# For which distributions is there a closed-form unbiased estimator for the standard deviation?

For the normal distribution, there is an unbiased estimator of the standard deviation given by:

$$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} \sqrt{\frac{1}{2}\sum_{k=1}^n(x_i-\bar{x})^2}$$

The reason this result is not so well known seems to be that it is largely a curio rather than a matter of any great import. The proof is covered on this thread; it takes advantage of a key property of the normal distribution:

$$\frac{1}{\sigma^2} \sum_{k=1}^n(x_i-\bar{x})^2 \sim \chi^{2}_{n-1}$$

From there, with a bit of work, it is possible to take the expectation $\mathbb{E}\left( \sqrt{\sum_{k=1}^n(x_i-\bar{x})^2} \right)$, and by identifying this answer as a multiple of $\sigma$, we can deduce the result for $\hat{\sigma}_\text{unbiased}$.

This leaves me curious which other distributions have a closed-form unbiased estimator of the standard deviation. Unlike with the unbiased estimator of the variance, this is clearly distribution-specific. Moreover, it would not be straightforward to adapt the proof to find estimators for other distributions.

The skew-normal distributions have some nice distributional properties for their quadratic forms, which the normal distribution property we used is effectively a special case of (since the normal is a special type of skew-normal) so perhaps it would not be so hard to extend this method to them. But for other distibutions it would appear an entirely different approach is required.

Are there any other distributions for which such estimators are known?

• If you ignore technical distractions, the nature of the answer becomes clearer. In the Normal case little of what you write is really relevant to the conclusion; all that matters is that the amount of bias in this particular estimator is a function of $n$ alone (and does not depend on other distributional parameters that need to be estimated from the data).
– whuber
Jan 5, 2015 at 23:49
• @whuber I think I can see the general idea you're hinting at, and clearly "function of $n$ alone" is necessary. But I don't think it would be sufficient - if we didn't have access to some nice distributional results, then I can't see how the "closed form" aspect would be tractable. Jan 5, 2015 at 23:56
• It depends on what you mean by "closed form." For instance, to one person a theta function may be "closed" but to another it's just an infinite product, power series, or complex integral. Come to think of it, that's precisely what a Gamma function is :-).
– whuber
Jan 6, 2015 at 0:00
• @whuber Good point! By "the amount of bias in this particular estimator", I take it you mean that the bias in $s$ (rather than the estimator listed in the question, which has zero bias) is a function of $n$ (and also in $\sigma$, but fortunately in such a way that we can easily rearrange to find an unbiased estimator)? Jan 6, 2015 at 0:13
• @whuber: There should be a similar formula for any location-scale family, with the caveat you pointed out that the function of $n$ may be an intractable integral. Jan 6, 2015 at 6:15

A bit late, but just recently became aware of the book

"V.G. Voinov and M.S. Nikulin, "Unbiased Estimators and Their Applications: Volume 1: Univariate Case", 1993"

which has unbiased estimators of functions of the parameters across many distributions in the appendix. The following is from the reference above, with their notation to be consistent and showing the true standard deviation with respect to their notation.

Normal Distribution

$$\sigma=\sigma_{\text{parameter}}$$ $$\widehat{\sigma}_{\text{unbiased}}= {\left[ \sqrt{2}\,\Gamma\left(\frac{n}{2}\right)\right]}^{-1} \sqrt{n-1}\, \Gamma\left(\frac{n-1}{2}\right) \sqrt{T}$$

where

$$T=\frac{1}{n-1} \sum_{i=1}^n {\left(X_i - \bar{X}\right)}^2$$

Uniform Distribution$$(0,\theta)$$ (continuous)

$$\sigma=\frac{\theta}{2\sqrt{3}}$$ $$\widehat{\sigma}_{\text{unbiased}}= \frac{(n+1) T}{2 \sqrt{3} n}$$

where $$T=\max\{X_i\}$$

Uniform Distribution$$(1,\theta)$$ (discrete)

$$\sigma=\frac{\sqrt{\theta^2-1}}{2\,\sqrt{3}} = g(\theta)$$ $$\widehat{\sigma}_{\text{unbiased}}=\frac{Y^n\,g(Y) - {(Y-1)}^n\,g(Y-1)}{Y^n - {(Y-1)}^n}$$

where $$Y=\max\{X_i\}$$

Gamma Distribution

$$\sigma=\sqrt{p}\,\theta$$ $$\widehat{\sigma}_{\text{unbiased}}=\frac{S}{n\,\sqrt{p}}$$

Where $$p$$ is known and $$S=\sum_{i=1}^n X_i$$

Exponential Distribution

Gamma Distribution with $$p=1$$

$$\sigma=\theta$$ $$\widehat{\sigma}_{\text{unbiased}}=\frac{S}{n}$$

Where $$S=\sum_{i=1}^n X_i$$

Inverse Gaussian distribution

$$\sigma=\sqrt{\frac{\mu^3}{\lambda}}$$

$$\lambda$$ known $$\widehat{\sigma}_{\text{unbiased}}=\frac{n^{1/4}\,S^{5/4}}{\lambda^{1/4}} \exp\left( \frac{n\,\lambda}{4\,S} \right)\,D_{-1/2}\left( \sqrt{\frac{n\,\lambda}{S}} \right)$$

$$\lambda , \mu$$ unknown

$$\widehat{\sigma}_{\text{unbiased}}= \frac{\sqrt{T\,S^3}\,\Gamma\left( \frac{n-1}{2} \right)}{\sqrt{2}\,\Gamma\left( \frac{n}{2} \right)} {}_2F_1 \left( \frac{3}{4}, \frac{1}{4} ; \frac{n}{2} ; -\frac{T\,S}{n} \right), \quad T\,S < n$$ $$\widehat{\sigma}_{\text{unbiased}}= \frac{\sqrt{T\,S^3}\,n^{3/4}\,\Gamma\left( \frac{n-1}{2} \right)}{\sqrt{2}\,{\left( n+ T\,S \right)}^{3/4}\,\Gamma\left( \frac{n}{2} \right)} {}_2F_1 \left( \frac{3}{4}, \frac{2\,n-1}{4} ; \frac{n}{2} ; -\frac{T\,S}{n+T\,S} \right), \quad T\,S \ge n, \quad n>1$$

where

$$S=\sum_{i=1}^n X_i / n$$ $$T=\sum_{i=1}^n \left( \frac{1}{X_i} - \frac{1}{S} \right)$$

Weibull distribution

$$p$$ is known

$$\sigma = \theta \sqrt{\Gamma \left(1+\frac{2}{p}\right)-\Gamma \left(1+\frac{1}{p}\right)^2}$$ $$\widehat{\sigma}_{\text{unbiased}}=\left( \frac{\Gamma(n)\,S^{1/p}}{\Gamma\left(n+\frac{1}{p}\right)} \right) \, \sqrt{\Gamma \left(1+\frac{2}{p}\right)-\Gamma\left(1+\frac{1}{p}\right)^2}, \quad n+\frac{1}{p}>0$$

where

$$S=\sum_{i=1}^n X_i^p$$

Rayleigh distribution

$$\sigma = \theta\,\sqrt{2-\frac{\pi}{2}}$$ $$\widehat{\sigma}_{\text{unbiased}}=\left( \frac{\Gamma(n)\,T^{1/2}}{\sqrt{2}\,\Gamma\left(n+\frac{1}{2}\right)} \right) \, \sqrt{2-\frac{\pi}{2}}$$

where

$$T=\sum_{i=1}^n X_i^2$$

Half-normal distribution

$$\sigma = \sigma_{\text{parameter}}\,\sqrt{1-\frac{2}{\pi}}$$ $$\widehat{\sigma}_{\text{unbiased}}=\left( \frac{\sqrt{T}\,\Gamma\left(\frac{n}{2}\right)}{\sqrt{2}\,\Gamma\left(\frac{n+1}{2}\right)} \right) \, \sqrt{1-\frac{2}{\pi}}$$

where

$$T=\sum_{i=1}^n X_i^2$$

Although this is not directly connected to the question, there is a 1968 paper by Peter Bickel and Erich Lehmann that states that, for a convex family of distributions $F$, there exists an unbiased estimator of a functional $q(F)$ (for a sample size $n$ large enough) if and only if $q(\alpha F+(1-\alpha)G)$ is a polynomial in $0\le \alpha\le 1$. This theorem does not apply to the problem here because the collection of Gaussian distributions is not convex (a mixture of Gaussians is not a Gaussian).

An extension of the result in the question is that any power $\sigma^\alpha$ of the standard deviation can be unbiasedly estimated, provided there are enough observations when $\alpha<0$. This follows from the result $$\frac{1}{\sigma^2} \sum_{k=1}^n(x_i-\bar{x})^2 \sim \chi^{2}_{n-1}$$ that $\sigma$ is the scale (and unique) parameter for $\sum_{k=1}^n(x_i-\bar{x})^2$.

This normal setting can then be extended to any location-scale family $$X_1,\ldots,X_n\stackrel{\text{iid}}{\sim} \tau^{-1}f(\tau^{-1}\{x-\mu\})$$ with a finite variance $\sigma^2$. Indeed,

1. the variance $$\text{var}_{\mu,\tau}(X)=\mathbb{E}_{\mu,\tau}[(X-\mu)^2]=\tau^2\mathbb{E}_{0,1}[X^2]$$ is only a function of $\tau$;
2. the sum of squares \begin{align*}\mathbb{E}_{\mu,\tau}\left[\sum_{k=1}^n(X_i-\bar{X})^2\right]&=\tau^2\mathbb{E}_{\mu,\tau}\left[\sum_{k=1}^n\tau^{-2}(X_i-\mu-\bar{X}+\mu)^2\right]\\ &=\tau^2\mathbb{E}_{0,1}\left[\sum_{k=1}^n(X_i-\bar{X})^2\right]\end{align*} has an expectation of the form $\tau^2\psi(n)$;
3. and similarly for any power $$\mathbb{E}_{\mu,\tau}\left[\left\{\sum_{k=1}^n(X_i-\bar{X})^2\right\}^\alpha\right]=\tau^{2\alpha}\mathbb{E}_{0,1}\left[\left\{\sum_{k=1}^n(X_i-\bar{X})^2\right\}^\alpha\right]$$ such that the expectation is finite.

A probably well known case, but a case nevertheless.
Consider a continuous uniform distribution $U(0,\theta)$. Given an i.i.d. sample, the maximum order statistic, $X_{(n)}$ has expected value

$$E(X_{(n)}) = \frac {n}{n+1}\theta$$

The standard deviation of the distribution is

$$\sigma = \frac {\theta}{2\sqrt 3}$$

So the estimator $$\hat \sigma = \frac 1{2\sqrt 3}\frac {n+1}{n}X_{(n)}$$

is evidently unbiased for $\sigma$.

This generalizes to the case where the lower bound of the distribution is also unknown, since we can have an unbiased estimator for the Range, and then the standard deviation is again a linear function of the Range (as is essentially above also).

This exemplifies @whuber's comment, that "the amount of bias is a function of $n$ alone" (plus possibly any known constants) -so it can be deterministically corrected. And this is the case here.

• Now the hard part: when in the world are we interested in the standard deviation of a uniform distribution? (+1) Jan 6, 2015 at 16:40
• @ssdecontrol That's an excellent question! -please proceed to the next one... Jan 6, 2015 at 17:10
• One thing I love about this answer is how poor the estimator is. It's quite common to see a question which boils down to "why do we use $\hat{\theta}$ as an estimator even though it's biased?" Some students need convincing that unbiasedness is not the be-all and end-all, and a poor unbiased estimator is one way to show them. Jan 9, 2015 at 17:14
• @Silverfish Poor in what way? Some quick simulations show this to have lower MSE than the usual standard deviation (which surprised me).
– Dave
Sep 27, 2019 at 16:47
• @Silverfish As you would know, $X_{(n)}$ is a complete sufficient statistic in this model. This in fact makes $\hat\sigma$ the minimum variance unbiased estimator of $\sigma$. Jan 10 at 17:52