61
$\begingroup$

What is an estimator of standard deviation of standard deviation if normality of data can be assumed?

Improve this question
$\endgroup$
1
  • $\begingroup$ I suppose that you are looking for the distribution of the sample variance. This links to a section on the Wikipedia page about variance on 16:55, 21 August 2016. Because this is a link to Wikipedia, the article might change in the future. Hence, the section may not reflect the contents this answer is referring to after such changes. Therefore a link to a historical version of the Wikipedia page is given here. The current article about variance is found [here](en.wikipedia.org/wik $\endgroup$
    – user28
    Commented Jul 26, 2010 at 18:40

3 Answers 3

Reset to default
66
$\begingroup$

Let $X_1, ..., X_n \sim N(\mu, \sigma^2)$. As shown in this thread, the standard deviation of the sample standard deviation,

$$ s = \sqrt{ \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \overline{X}) }, $$

is

$$ {\rm SD}(s) = \sqrt{ E \left( [E(s)- s]^2 \right) } = \sigma \sqrt{ 1 - \frac{2}{n-1} \cdot \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \right)^2 } $$

where $\Gamma(\cdot)$ is the gamma function, $n$ is the sample size and $\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$ is the sample mean. Since $s$ is a consistent estimator of $\sigma$, this suggests replacing $\sigma$ with $s$ in the equation above to get a consistent estimator of ${\rm SD}(s)$.

If it is an unbiased estimator you seek, we see in this thread that $ E(s) = \sigma \cdot \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } $, which, by linearity of expectation, suggests

$$ s \cdot \sqrt{ \frac{n-1}{2} } \cdot \frac{\Gamma( \frac{n-1}{2} )}{ \Gamma(n/2) } $$

as an unbiased estimator of $\sigma$. All of this together with linearity of expectation gives an unbiased estimator of ${\rm SD}(s)$:

$$ s \cdot \frac{\Gamma( \frac{n-1}{2} )}{ \Gamma(n/2) } \cdot \sqrt{\frac{n-1}{2} - \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \right)^2 } $$

Improve this answer
$\endgroup$
7
  • 15
    $\begingroup$ +1 It's nice to see not only a better reply come along after almost two years, but a reply that provides more useful detail than the references elsewhere in this thread. $\endgroup$
    – whuber ♦
    Commented May 16, 2012 at 21:55
  • 3
    $\begingroup$ Did you forget to square the distances in the first formula? $\endgroup$
    – danijar
    Commented Aug 9, 2016 at 19:10
  • 5
    $\begingroup$ The Gamma function is hard to calculate for non-small values of $n$. Applying Stirling's approximation, I get $s\cdot\sqrt{\mathrm{e}\cdot(1-\frac{1}{n})^{n-1}-1}$, which is computationally feasible as well as a bit more compact expression-wise. $\endgroup$
    – equaeghe
    Commented Nov 3, 2016 at 18:57
  • 1
    $\begingroup$ Probably worth pointing out that s (computed in @Macro's answer is sometimes referred to as the standard error of the sample standard deviation. $\endgroup$
    – Harvey Motulsky
    Commented Feb 4, 2017 at 16:18
  • 1
    $\begingroup$ For those who want a simple form, $s/\sqrt{2(n-1)}$ is a good approximation at a few percent level. $\endgroup$
    – Syrtis Major
    Commented Jul 18, 2019 at 5:01
6
$\begingroup$

Assume you observe $X_1,\dots,X_n$ iid from a normal with mean zero and variance $\sigma^2$. The (empirical) standard deviation is the square root of the estimator $\hat{\sigma}^2$ of $\sigma^2$ (unbiased or not that is not the question). As an estimator (obtained with $X_1,\dots,X_n$), $\hat{\sigma}$ has a variance that can be calculated theoretically. Maybe what you call the standard deviation of standard deviation is actually the square root of the variance of the standard deviation, i.e. $\sqrt{E[(\sigma-\hat{\sigma})^2]}$? It is not an estimator, it is a theoretical quantity (something like $\sigma/\sqrt{n}$ to be confirmed) that can be calculated explicitely !

Improve this answer
$\endgroup$
4
  • $\begingroup$ Isn't that a function of estimator is still an estimator? I still don't know \sigma, only X_i. $\endgroup$
    – user88
    Commented Jul 26, 2010 at 16:45
  • $\begingroup$ ok, then you will possibly estimate the square root of the variance of the estimation of the square root of the variance... right :) should be something like $\hat{\sigma}/n$ ? $\endgroup$
    – robin girard
    Commented Jul 26, 2010 at 17:09
  • $\begingroup$ What Srikant found (and what seems confirmed at PhysicsForums) there should be $\sqrt{2}$, so rather $\hat{\sigma}\frac{\sqrt{2}}{2n}$. $\endgroup$
    – user88
    Commented Aug 1, 2010 at 15:34
  • 1
    $\begingroup$ Aww, those comments locks; $\frac{\hat{\sigma}}{\sqrt{2n}}$. At least this one gives the result in agreement with bootstrap. $\endgroup$
    – user88
    Commented Aug 1, 2010 at 15:58
-3
$\begingroup$

@Macro provided a great mathematical explanation with equation to compute. Here is a more general explation for less mathematical people.

I think the terminology "SD of SD" is confusing to many. It is easier to think about the confidence interval of a SD. How precise is the standard deviation you compute from a sample? Just by chance you may have happened to obtain data that are closely bunched together, making the sample SD much lower than the population SD. Or you may have randomly obtained values that are far more scattered than the overall population, making the sample SD higher than the population SD.

Interpreting the CI of the SD is straightforward. Start with the customary assumption that your data were randomly and independently sampled from a Gaussian distribution. Now repeat this sampling many times. You expect 95% of those confidence intervals to include the true population SD.

How wide is the 95% confidence interval of a SD? It depends on sample size (n) of course.

n: 95% CI of SD

2: 0.45*SD to 31.9*SD

3: 0.52*SD to 6.29*SD

5: 0.60*SD to 2.87*SD

10: 0.69*SD to 1.83*SD

25: 0.78*SD to 1.39*SD

50: 0.84*SD to 1.25*SD

100: 0.88*SD to 1.16*SD

500: 0.94*SD to 1.07*SD

Free web calculator

Improve this answer
$\endgroup$
5
  • $\begingroup$ I can do Monte Carlo, I just wanted to do in a more 'sciency' way; still you're right that the distribution is not normal, so this sd will be useless for testing. $\endgroup$
    – user88
    Commented Jul 26, 2010 at 19:50
  • 4
    $\begingroup$ For what it's worth, I'm uncomfortable with the statement "a confidence interval that is 95%... likely to contain the true SD" (or, stated more explicitly in the linked page: "you can be 95% sure that the CI computed from the sample SD contains the true population SD"). I think these statements flirt w/ reinforcing a popular misconception, see here, eg, for a related discussion on CV. $\endgroup$
    – gung - Reinstate Monica
    Commented May 17, 2012 at 16:37
  • 5
    $\begingroup$ What is "I think both the concept and the terminology of "SD of SD" is too slippery to tackle" supposed to mean? The sample standard deviation is a random variable that has a standard deviation. $\endgroup$
    – Macro
    Commented Jun 8, 2012 at 12:18
  • $\begingroup$ @Macro. Thanks for your comments. I rewrote substantially. $\endgroup$
    – Harvey Motulsky
    Commented Feb 4, 2017 at 16:11
  • 1
    $\begingroup$ @gung . I rewrote to properly explain the confidence interval. $\endgroup$
    – Harvey Motulsky
    Commented Feb 4, 2017 at 16:12