52
$\begingroup$

What is an estimator of standard deviation of standard deviation if normality of data can be assumed?

$\endgroup$
56
$\begingroup$

Let $X_1, ..., X_n \sim N(\mu, \sigma^2)$. As shown in this thread, the standard deviation of the sample standard deviation,

$$ s = \sqrt{ \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \overline{X}) }, $$

is

$$ {\rm SD}(s) = \sqrt{ E \left( [E(s)- s]^2 \right) } = \sigma \sqrt{ 1 - \frac{2}{n-1} \cdot \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \right)^2 } $$

where $\Gamma(\cdot)$ is the gamma function, $n$ is the sample size and $\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$ is the sample mean. Since $s$ is a consistent estimator of $\sigma$, this suggests replacing $\sigma$ with $s$ in the equation above to get a consistent estimator of ${\rm SD}(s)$.

If it is an unbiased estimator you seek, we see in this thread that $ E(s) = \sigma \cdot \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } $, which, by linearity of expectation, suggests

$$ s \cdot \sqrt{ \frac{n-1}{2} } \cdot \frac{\Gamma( \frac{n-1}{2} )}{ \Gamma(n/2) } $$

as an unbiased estimator of $\sigma$. All of this together with linearity of expectation gives an unbiased estimator of ${\rm SD}(s)$:

$$ s \cdot \frac{\Gamma( \frac{n-1}{2} )}{ \Gamma(n/2) } \cdot \sqrt{\frac{n-1}{2} - \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \right)^2 } $$

$\endgroup$
  • 12
    $\begingroup$ +1 It's nice to see not only a better reply come along after almost two years, but a reply that provides more useful detail than the references elsewhere in this thread. $\endgroup$ – whuber May 16 '12 at 21:55
  • 2
    $\begingroup$ Did you forget to square the distances in the first formula? $\endgroup$ – danijar Aug 9 '16 at 19:10
  • 2
    $\begingroup$ The Gamma function is hard to calculate for non-small values of $n$. Applying Stirling's approximation, I get $s\cdot\sqrt{\mathrm{e}\cdot(1-\frac{1}{n})^{n-1}-1}$, which is computationally feasible as well as a bit more compact expression-wise. $\endgroup$ – equaeghe Nov 3 '16 at 18:57
  • 1
    $\begingroup$ Probably worth pointing out that s (computed in @Macro's answer is sometimes referred to as the standard error of the sample standard deviation. $\endgroup$ – Harvey Motulsky Feb 4 '17 at 16:18
  • $\begingroup$ For those who want a simple form, $s/\sqrt{2(n-1)}$ is a good approximation at a few percent level. $\endgroup$ – Syrtis Major Jul 18 at 5:01
14
$\begingroup$

I suppose that you are looking for the distribution of the sample variance. This links to a section on the Wikipedia page about variance on 16:55, 21 August 2016. Because this is a link to Wikipedia, the article might change in the future. Hence, the section may not reflect the contents this answer is referring to after such changes. Therefore a link to a historical version of the Wikipedia page is given here.

The current article about variance is found here.

$\endgroup$

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • $\begingroup$ This should really be a comment. $\endgroup$ – Silverfish Oct 29 '15 at 0:34
  • $\begingroup$ @Silverfish, it should have been originally, but might be worth grandfathering in this case. $\endgroup$ – gung Oct 29 '15 at 0:57
  • 2
    $\begingroup$ @gung It is a link to a Wikipedia section rather than a page, and these are especially vulnerable to link rot (because an edit to the wiki article - e.g. retitling the section to "Sampling distribution" or similar - can lead to the link simply pointing to the "variance" article, which wouldn't add much value for a future reader). I suppose one advantage to leaving this as an answer rather than a comment is that if any future user notices such a change, they might edit this answer. $\endgroup$ – Silverfish Oct 29 '15 at 1:04
5
$\begingroup$

Assume you observe $X_1,\dots,X_n$ iid from a normal with mean zero and variance $\sigma^2$. The (empirical) standard deviation is the square root of the estimator $\hat{\sigma}^2$ of $\sigma^2$ (unbiased or not that is not the question). As an estimator (obtained with $X_1,\dots,X_n$), $\hat{\sigma}$ has a variance that can be calculated theoretically. Maybe what you call the standard deviation of standard deviation is actually the square root of the variance of the standard deviation, i.e. $\sqrt{E[(\sigma-\hat{\sigma})^2]}$? It is not an estimator, it is a theoretical quantity (something like $\sigma/\sqrt{n}$ to be confirmed) that can be calculated explicitely !

$\endgroup$
  • $\begingroup$ Isn't that a function of estimator is still an estimator? I still don't know \sigma, only X_i. $\endgroup$ – user88 Jul 26 '10 at 16:45
  • $\begingroup$ ok, then you will possibly estimate the square root of the variance of the estimation of the square root of the variance... right :) should be something like $\hat{\sigma}/n$ ? $\endgroup$ – robin girard Jul 26 '10 at 17:09
  • $\begingroup$ What Srikant found (and what seems confirmed at PhysicsForums) there should be $\sqrt{2}$, so rather $\hat{\sigma}\frac{\sqrt{2}}{2n}$. $\endgroup$ – user88 Aug 1 '10 at 15:34
  • 1
    $\begingroup$ Aww, those comments locks; $\frac{\hat{\sigma}}{\sqrt{2n}}$. At least this one gives the result in agreement with bootstrap. $\endgroup$ – user88 Aug 1 '10 at 15:58
-3
$\begingroup$

@Macro provided a great mathematical explanation with equation to compute. Here is a more general explation for less mathematical people.

I think the terminology "SD of SD" is confusing to many. It is easier to think about the confidence interval of a SD. How precise is the standard deviation you compute from a sample? Just by chance you may have happened to obtain data that are closely bunched together, making the sample SD much lower than the population SD. Or you may have randomly obtained values that are far more scattered than the overall population, making the sample SD higher than the population SD.

Interpreting the CI of the SD is straightforward. Start with the customary assumption that your data were randomly and independently sampled from a Gaussian distribution. Now repeat this sampling many times. You expect 95% of those confidence intervals to include the true population SD.

How wide is the 95% confidence interval of a SD? It depends on sample size (n) of course.

n: 95% CI of SD

2: 0.45*SD to 31.9*SD

3: 0.52*SD to 6.29*SD

5: 0.60*SD to 2.87*SD

10: 0.69*SD to 1.83*SD

25: 0.78*SD to 1.39*SD

50: 0.84*SD to 1.25*SD

100: 0.88*SD to 1.16*SD

500: 0.94*SD to 1.07*SD

Free web calculator

$\endgroup$
  • $\begingroup$ I can do Monte Carlo, I just wanted to do in a more 'sciency' way; still you're right that the distribution is not normal, so this sd will be useless for testing. $\endgroup$ – user88 Jul 26 '10 at 19:50
  • 4
    $\begingroup$ For what it's worth, I'm uncomfortable with the statement "a confidence interval that is 95%... likely to contain the true SD" (or, stated more explicitly in the linked page: "you can be 95% sure that the CI computed from the sample SD contains the true population SD"). I think these statements flirt w/ reinforcing a popular misconception, see here, eg, for a related discussion on CV. $\endgroup$ – gung May 17 '12 at 16:37
  • 5
    $\begingroup$ What is "I think both the concept and the terminology of "SD of SD" is too slippery to tackle" supposed to mean? The sample standard deviation is a random variable that has a standard deviation. $\endgroup$ – Macro Jun 8 '12 at 12:18
  • $\begingroup$ @Macro. Thanks for your comments. I rewrote substantially. $\endgroup$ – Harvey Motulsky Feb 4 '17 at 16:11
  • 1
    $\begingroup$ @gung . I rewrote to properly explain the confidence interval. $\endgroup$ – Harvey Motulsky Feb 4 '17 at 16:12

protected by kjetil b halvorsen Dec 14 '17 at 20:53

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?