For which distributions is there a closed-form unbiased estimator for the standard deviation?

Question

For the normal distribution, there is an unbiased estimator of the standard deviation given by:

$$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} \sqrt{\frac{1}{2}\sum_{k=1}^n(x_i-\bar{x})^2}$$

The reason this result is not so well known seems to be that it is largely a curio rather than a matter of any great import. The proof is covered on this thread; it takes advantage of a key property of the normal distribution:

$$ \frac{1}{\sigma^2} \sum_{k=1}^n(x_i-\bar{x})^2 \sim \chi^{2}_{n-1} $$

From there, with a bit of work, it is possible to take the expectation $\mathbb{E}\left( \sqrt{\sum_{k=1}^n(x_i-\bar{x})^2} \right)$, and by identifying this answer as a multiple of $\sigma$, we can deduce the result for $\hat{\sigma}_\text{unbiased}$.

This leaves me curious which other distributions have a closed-form unbiased estimator of the standard deviation. Unlike with the unbiased estimator of the variance, this is clearly distribution-specific. Moreover, it would not be straightforward to adapt the proof to find estimators for other distributions.

The skew-normal distributions have some nice distributional properties for their quadratic forms, which the normal distribution property we used is effectively a special case of (since the normal is a special type of skew-normal) so perhaps it would not be so hard to extend this method to them. But for other distibutions it would appear an entirely different approach is required.

Are there any other distributions for which such estimators are known?

Answer 1

A bit late, but just recently became aware of the book

"V.G. Voinov and M.S. Nikulin, "Unbiased Estimators and Their Applications: Volume 1: Univariate Case", 1993"

which has unbiased estimators of functions of the parameters across many distributions in the appendix. The following is from the reference above, with their notation to be consistent and showing the true standard deviation with respect to their notation.

Normal Distribution

$$\sigma=\sigma_{\text{parameter}}$$ $$\widehat{\sigma}_{\text{unbiased}}= {\left[ \sqrt{2}\,\Gamma\left(\frac{n}{2}\right)\right]}^{-1} \sqrt{n-1}\, \Gamma\left(\frac{n-1}{2}\right) \sqrt{T} $$

where

$$T=\frac{1}{n-1} \sum_{i=1}^n {\left(X_i - \bar{X}\right)}^2$$

Uniform Distribution$(0,\theta)$ (continuous)

$$\sigma=\frac{\theta}{2\sqrt{3}}$$ $$\widehat{\sigma}_{\text{unbiased}}= \frac{(n+1) T}{2 \sqrt{3} n}$$

where $T=\max\{X_i\}$

Uniform Distribution$(1,\theta)$ (discrete)

$$\sigma=\frac{\sqrt{\theta^2-1}}{2\,\sqrt{3}} = g(\theta)$$ $$\widehat{\sigma}_{\text{unbiased}}=\frac{Y^n\,g(Y) - {(Y-1)}^n\,g(Y-1)}{Y^n - {(Y-1)}^n}$$

where $Y=\max\{X_i\}$

Gamma Distribution

$$\sigma=\sqrt{p}\,\theta$$ $$\widehat{\sigma}_{\text{unbiased}}=\frac{S}{n\,\sqrt{p}}$$

Where $p$ is known and $S=\sum_{i=1}^n X_i$

Exponential Distribution

Gamma Distribution with $p=1$

$$\sigma=\theta$$ $$\widehat{\sigma}_{\text{unbiased}}=\frac{S}{n}$$

Where $S=\sum_{i=1}^n X_i$

Inverse Gaussian distribution

$$\sigma=\sqrt{\frac{\mu^3}{\lambda}}$$

$\lambda$ known $$\widehat{\sigma}_{\text{unbiased}}=\frac{n^{1/4}\,S^{5/4}}{\lambda^{1/4}} \exp\left( \frac{n\,\lambda}{4\,S} \right)\,D_{-1/2}\left( \sqrt{\frac{n\,\lambda}{S}} \right) $$

$\lambda , \mu $ unknown

$$\widehat{\sigma}_{\text{unbiased}}= \frac{\sqrt{T\,S^3}\,\Gamma\left( \frac{n-1}{2} \right)}{\sqrt{2}\,\Gamma\left( \frac{n}{2} \right)} {}_2F_1 \left( \frac{3}{4}, \frac{1}{4} ; \frac{n}{2} ; -\frac{T\,S}{n} \right), \quad T\,S < n$$ $$\widehat{\sigma}_{\text{unbiased}}= \frac{\sqrt{T\,S^3}\,n^{3/4}\,\Gamma\left( \frac{n-1}{2} \right)}{\sqrt{2}\,{\left( n+ T\,S \right)}^{3/4}\,\Gamma\left( \frac{n}{2} \right)} {}_2F_1 \left( \frac{3}{4}, \frac{2\,n-1}{4} ; \frac{n}{2} ; -\frac{T\,S}{n+T\,S} \right), \quad T\,S \ge n, \quad n>1$$

where

$$S=\sum_{i=1}^n X_i / n$$ $$T=\sum_{i=1}^n \left( \frac{1}{X_i} - \frac{1}{S} \right) $$

Weibull distribution

$p$ is known

$$\sigma = \theta \sqrt{\Gamma \left(1+\frac{2}{p}\right)-\Gamma \left(1+\frac{1}{p}\right)^2}$$ $$\widehat{\sigma}_{\text{unbiased}}=\left( \frac{\Gamma(n)\,S^{1/p}}{\Gamma\left(n+\frac{1}{p}\right)} \right) \, \sqrt{\Gamma \left(1+\frac{2}{p}\right)-\Gamma\left(1+\frac{1}{p}\right)^2}, \quad n+\frac{1}{p}>0 $$

where

$$S=\sum_{i=1}^n X_i^p $$

Rayleigh distribution

$$\sigma = \theta\,\sqrt{2-\frac{\pi}{2}} $$ $$\widehat{\sigma}_{\text{unbiased}}=\left( \frac{\Gamma(n)\,T^{1/2}}{\sqrt{2}\,\Gamma\left(n+\frac{1}{2}\right)} \right) \, \sqrt{2-\frac{\pi}{2}} $$

where

$$T=\sum_{i=1}^n X_i^2 $$

Half-normal distribution

$$\sigma = \sigma_{\text{parameter}}\,\sqrt{1-\frac{2}{\pi}} $$ $$\widehat{\sigma}_{\text{unbiased}}=\left( \frac{\sqrt{T}\,\Gamma\left(\frac{n}{2}\right)}{\sqrt{2}\,\Gamma\left(\frac{n+1}{2}\right)} \right) \, \sqrt{1-\frac{2}{\pi}} $$

where

$$T=\sum_{i=1}^n X_i^2 $$

Answer 2

Although this is not directly connected to the question, there is a 1968 paper by Peter Bickel and Erich Lehmann that states that, for a convex family of distributions $F$, there exists an unbiased estimator of a functional $q(F)$ (for a sample size $n$ large enough) if and only if $q(\alpha F+(1-\alpha)G)$ is a polynomial in $0\le \alpha\le 1$. This theorem does not apply to the problem here because the collection of Gaussian distributions is not convex (a mixture of Gaussians is not a Gaussian).

An extension of the result in the question is that any power $\sigma^\alpha$ of the standard deviation can be unbiasedly estimated, provided there are enough observations when $\alpha<0$. This follows from the result $$\frac{1}{\sigma^2} \sum_{k=1}^n(x_i-\bar{x})^2 \sim \chi^{2}_{n-1}$$ that $\sigma$ is the scale (and unique) parameter for $\sum_{k=1}^n(x_i-\bar{x})^2$.

This normal setting can then be extended to any location-scale family $$X_1,\ldots,X_n\stackrel{\text{iid}}{\sim} \tau^{-1}f(\tau^{-1}\{x-\mu\})$$ with a finite variance $\sigma^2$. Indeed,

1. the variance $$\text{var}_{\mu,\tau}(X)=\mathbb{E}_{\mu,\tau}[(X-\mu)^2]=\tau^2\mathbb{E}_{0,1}[X^2]$$ is only a function of $\tau$;
2. the sum of squares \begin{align*}\mathbb{E}_{\mu,\tau}\left[\sum_{k=1}^n(X_i-\bar{X})^2\right]&=\tau^2\mathbb{E}_{\mu,\tau}\left[\sum_{k=1}^n\tau^{-2}(X_i-\mu-\bar{X}+\mu)^2\right]\\ &=\tau^2\mathbb{E}_{0,1}\left[\sum_{k=1}^n(X_i-\bar{X})^2\right]\end{align*} has an expectation of the form $\tau^2\psi(n)$;
3. and similarly for any power $$\mathbb{E}_{\mu,\tau}\left[\left\{\sum_{k=1}^n(X_i-\bar{X})^2\right\}^\alpha\right]=\tau^{2\alpha}\mathbb{E}_{0,1}\left[\left\{\sum_{k=1}^n(X_i-\bar{X})^2\right\}^\alpha\right]$$ such that the expectation is finite.

Answer 3

A probably well known case, but a case nevertheless.
Consider a continuous uniform distribution $U(0,\theta)$. Given an i.i.d. sample, the maximum order statistic, $X_{(n)}$ has expected value

$$E(X_{(n)}) = \frac {n}{n+1}\theta $$

The standard deviation of the distribution is

$$\sigma = \frac {\theta}{2\sqrt 3}$$

So the estimator $$\hat \sigma = \frac 1{2\sqrt 3}\frac {n+1}{n}X_{(n)}$$

is evidently unbiased for $\sigma$.

This generalizes to the case where the lower bound of the distribution is also unknown, since we can have an unbiased estimator for the Range, and then the standard deviation is again a linear function of the Range (as is essentially above also).

This exemplifies @whuber's comment, that "the amount of bias is a function of $n$ alone" (plus possibly any known constants) -so it can be deterministically corrected. And this is the case here.