4
$\begingroup$

Suppose we have $k$ groups of $n$ i.i.d observations which follow the $N(0,1)$ parent distribution. The sample standard deviation $S$ is a random variable (with $k$ available observations), do we know what distribution does it follow? I also have the same question when the parent distribution is $N^2(0,1)$, which basically means that the parent distribution is a $\chi^2_1$.

$\endgroup$
1
  • 1
    $\begingroup$ I just upvoted the answer by @BruceET, but this might be of interest if you actually want the somewhat ugly distribution: stats.stackexchange.com/a/410311/247352. $\endgroup$
    – Ed V
    Apr 16, 2020 at 20:29

2 Answers 2

1
$\begingroup$

The relevant distribution here is called the chi distribution:

$$S \sim \frac{\sigma}{\sqrt{n-1}} \cdot \text{Chi}(\text{df} = n-1).$$

Using the rules for transformations of random variables, the density function for the standard deviation is:

$$\begin{aligned} f_S(s) &= \text{Chi} \Bigg( \frac{\sqrt{n-1} \cdot s}{\sigma} \Bigg| \text{df} = n-1 \Bigg) \cdot \frac{\sqrt{n-1}}{\sigma} \\[6pt] &= \frac{(n-1)^{n/2}}{2^{(n-3)/2} \cdot \sigma \cdot \Gamma(\tfrac{n-1}{2})} \cdot \Big( \frac{s}{\sigma} \Big)^{n-2} \cdot \exp \Big( - \frac{n-1}{2} \cdot \frac{s^2}{\sigma^2} \Big). \\[6pt] \end{aligned}$$

The resulting mean and variance are:

$$\begin{aligned} \mathbb{E}(S) &= \sigma \cdot \sqrt{\frac{2}{n-1}} \cdot \frac{\Gamma(\tfrac{n}{2})}{\Gamma(\tfrac{n-1}{2})}, \\[6pt] \mathbb{V}(S) &= \sigma^2 \Bigg[ 1 - \frac{2}{n-1} \cdot \frac{\Gamma(\tfrac{n}{2})^2}{\Gamma(\tfrac{n-1}{2})^2} \Bigg]. \\[6pt] \end{aligned}$$

$\endgroup$
2
  • $\begingroup$ Thanks for the answer. Is it possible to derive the mean and the variance of S from this p.d.f? (calculus problem) Also, what happens when the parent distribution is from a $X^2_1$ distribution? $\endgroup$ Apr 17, 2020 at 12:40
  • 1
    $\begingroup$ From quality control books, I believe that the mean of S which is described by the p.d.f you mentioned above is $c_4*\sigma$ and the variance is $\sigma^2 * (1-c_4)$ but I do not know why. Does the p.d.f. above have a name? I am interested in deriving the c.d.f $\endgroup$ Apr 17, 2020 at 12:49
3
$\begingroup$

It is easier to discuss the distribution of the sample variance $$S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2,$$ where $X_i$ are a random sample from $\mathsf{Norm}(\mu, \sigma)$ and $\bar X$ is the sample mean. In that case, $$\frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(\nu = n-1).$$ Thus, the distribution of $S^2$ is a multiple of $\mathsf{Chisq}(\nu = n-1).$ The second displayed relationship can be proved using a multivariate transformaion or by using moment generating functions. [After an appropriate linear transformation of an $n$-dimensional multivariate normal distribution, $\bar X$ has a one dimensional marginal distribution and, independently, $S^2$ is a function of $n-1$ dimensions.]

You don't say why you seek the distribution of $S.$ The relationship above, using $\mathsf{Chisq}(\nu = n-1),$ can be used to find a 95% confidence interval for $\sigma$ as follows.

First, a 95% CI for $\sigma^2$ is of the form $\left(\frac{(n-1)S^2}{U},\frac{(n-1)S^2}{L}\right),$ where $L$ and $U$ cut probabiltiy 0.025 from the left and right tails, respectively, of $\mathsf{Chisq}(\nu = n-1).$ Then, to find a 95% CI for $\sigma,$ take square roots of the endpoints of the above CI for $\sigma^2.$

Of course, the distribution of $S$ can be found by taking the square root of the relevant chi-squared distribution. But historically, printed tables of percentage points of chi-squared distributions have been available, so the distribution of $S^2$ is more commonly used.

Notes: (1) For normal data $E(S^2_n) = \sigma^2,$ but because of the linear nature of expectation, this equality does not survive taking square roots: $E(S_n) = \sigma\sqrt{\frac{2}{n-1}}\Gamma(\frac{n}{2})/\Gamma(\frac{n-1}{2}) < \sigma,$ where $S_n$ is the standard deviation of a sample of size $n.$ The bias of $S_n$ as an estimator of $\sigma$ is small; except for very small samples. For samples of moderate or large size, the bias is often ignored in practice. For example: $E(S_5) = 0.9400\sigma$ and $E(S_{50}) = 0.9949\sigma.$

n = c(5,25,50);  round(sqrt((2/(n-1)))*gamma(n/2)/gamma((n-1)/2),4)
[1] 0.9400 0.9896 0.9949

(2) For a random sample from the standard normal distribution, the distribution of $S_{25}$ is simulated below:

set.seed(2020)
s = replicate( 10^6, sd(rnorm(25)) )
summary(s)
  Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
0.4191  0.8905  0.9862  0.9897  1.0850  1.7311 
hist(s, prob=T, br=30, col="skyblue2", main="Dist'n of Sample SD")

enter image description here

$\endgroup$
2
  • $\begingroup$ I upvoted your nice answer, but tiny thing: second equal sign in first equation. $\endgroup$
    – Ed V
    Apr 16, 2020 at 20:31
  • $\begingroup$ @EdV. Thanks for pointing that out. Fixed that and a couple of other mor minor typos just now. Hope I've found them all. $\endgroup$
    – BruceET
    Apr 16, 2020 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.