Sampe Standard deviation, what distribution does it follow?

Question

Suppose we have $k$ groups of $n$ i.i.d observations which follow the $N(0,1)$ parent distribution. The sample standard deviation $S$ is a random variable (with $k$ available observations), do we know what distribution does it follow? I also have the same question when the parent distribution is $N^2(0,1)$, which basically means that the parent distribution is a $\chi^2_1$.

Answer 1

The relevant distribution here is called the chi distribution:

$$S \sim \frac{\sigma}{\sqrt{n-1}} \cdot \text{Chi}(\text{df} = n-1).$$

Using the rules for transformations of random variables, the density function for the standard deviation is:

$$\begin{aligned} f_S(s) &= \text{Chi} \Bigg( \frac{\sqrt{n-1} \cdot s}{\sigma} \Bigg| \text{df} = n-1 \Bigg) \cdot \frac{\sqrt{n-1}}{\sigma} \\[6pt] &= \frac{(n-1)^{n/2}}{2^{(n-3)/2} \cdot \sigma \cdot \Gamma(\tfrac{n-1}{2})} \cdot \Big( \frac{s}{\sigma} \Big)^{n-2} \cdot \exp \Big( - \frac{n-1}{2} \cdot \frac{s^2}{\sigma^2} \Big). \\[6pt] \end{aligned}$$

The resulting mean and variance are:

$$\begin{aligned} \mathbb{E}(S) &= \sigma \cdot \sqrt{\frac{2}{n-1}} \cdot \frac{\Gamma(\tfrac{n}{2})}{\Gamma(\tfrac{n-1}{2})}, \\[6pt] \mathbb{V}(S) &= \sigma^2 \Bigg[ 1 - \frac{2}{n-1} \cdot \frac{\Gamma(\tfrac{n}{2})^2}{\Gamma(\tfrac{n-1}{2})^2} \Bigg]. \\[6pt] \end{aligned}$$

Answer 2

It is easier to discuss the distribution of the sample variance $$S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2,$$ where $X_i$ are a random sample from $\mathsf{Norm}(\mu, \sigma)$ and $\bar X$ is the sample mean. In that case, $$\frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(\nu = n-1).$$ Thus, the distribution of $S^2$ is a multiple of $\mathsf{Chisq}(\nu = n-1).$ The second displayed relationship can be proved using a multivariate transformaion or by using moment generating functions. [After an appropriate linear transformation of an $n$-dimensional multivariate normal distribution, $\bar X$ has a one dimensional marginal distribution and, independently, $S^2$ is a function of $n-1$ dimensions.]

You don't say why you seek the distribution of $S.$ The relationship above, using $\mathsf{Chisq}(\nu = n-1),$ can be used to find a 95% confidence interval for $\sigma$ as follows.

First, a 95% CI for $\sigma^2$ is of the form $\left(\frac{(n-1)S^2}{U},\frac{(n-1)S^2}{L}\right),$ where $L$ and $U$ cut probabiltiy 0.025 from the left and right tails, respectively, of $\mathsf{Chisq}(\nu = n-1).$ Then, to find a 95% CI for $\sigma,$ take square roots of the endpoints of the above CI for $\sigma^2.$

Of course, the distribution of $S$ can be found by taking the square root of the relevant chi-squared distribution. But historically, printed tables of percentage points of chi-squared distributions have been available, so the distribution of $S^2$ is more commonly used.

Notes: (1) For normal data $E(S^2_n) = \sigma^2,$ but because of the linear nature of expectation, this equality does not survive taking square roots: $E(S_n) = \sigma\sqrt{\frac{2}{n-1}}\Gamma(\frac{n}{2})/\Gamma(\frac{n-1}{2}) < \sigma,$ where $S_n$ is the standard deviation of a sample of size $n.$ The bias of $S_n$ as an estimator of $\sigma$ is small; except for very small samples. For samples of moderate or large size, the bias is often ignored in practice. For example: $E(S_5) = 0.9400\sigma$ and $E(S_{50}) = 0.9949\sigma.$

n = c(5,25,50);  round(sqrt((2/(n-1)))*gamma(n/2)/gamma((n-1)/2),4)
[1] 0.9400 0.9896 0.9949

(2) For a random sample from the standard normal distribution, the distribution of $S_{25}$ is simulated below:

set.seed(2020)
s = replicate( 10^6, sd(rnorm(25)) )
summary(s)
  Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
0.4191  0.8905  0.9862  0.9897  1.0850  1.7311 
hist(s, prob=T, br=30, col="skyblue2", main="Dist'n of Sample SD")