16
$\begingroup$

For the normal distribution, there is an unbiased estimator of the standard deviation given by:

$$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} \sqrt{\frac{1}{2}\sum_{k=1}^n(x_i-\bar{x})^2}$$

The reason this result is not so well known seems to be that it is largely a curio rather than a matter of any great import. The proof is covered on this thread; it takes advantage of a key property of the normal distribution:

$$ \frac{1}{\sigma^2} \sum_{k=1}^n(x_i-\bar{x})^2 \sim \chi^{2}_{n-1} $$

From there, with a bit of work, it is possible to take the expectation $\mathbb{E}\left( \sqrt{\sum_{k=1}^n(x_i-\bar{x})^2} \right)$, and by identifying this answer as a multiple of $\sigma$, we can deduce the result for $\hat{\sigma}_\text{unbiased}$.

This leaves me curious which other distributions have a closed-form unbiased estimator of the standard deviation. Unlike with the unbiased estimator of the variance, this is clearly distribution-specific. Moreover, it would not be straightforward to adapt the proof to find estimators for other distributions.

The skew-normal distributions have some nice distributional properties for their quadratic forms, which the normal distribution property we used is effectively a special case of (since the normal is a special type of skew-normal) so perhaps it would not be so hard to extend this method to them. But for other distibutions it would appear an entirely different approach is required.

Are there any other distributions for which such estimators are known?

$\endgroup$
  • 1
    $\begingroup$ If you ignore technical distractions, the nature of the answer becomes clearer. In the Normal case little of what you write is really relevant to the conclusion; all that matters is that the amount of bias in this particular estimator is a function of $n$ alone (and does not depend on other distributional parameters that need to be estimated from the data). $\endgroup$ – whuber Jan 5 '15 at 23:49
  • $\begingroup$ @whuber I think I can see the general idea you're hinting at, and clearly "function of $n$ alone" is necessary. But I don't think it would be sufficient - if we didn't have access to some nice distributional results, then I can't see how the "closed form" aspect would be tractable. $\endgroup$ – Silverfish Jan 5 '15 at 23:56
  • 5
    $\begingroup$ It depends on what you mean by "closed form." For instance, to one person a theta function may be "closed" but to another it's just an infinite product, power series, or complex integral. Come to think of it, that's precisely what a Gamma function is :-). $\endgroup$ – whuber Jan 6 '15 at 0:00
  • $\begingroup$ @whuber Good point! By "the amount of bias in this particular estimator", I take it you mean that the bias in $s$ (rather than the estimator listed in the question, which has zero bias) is a function of $n$ (and also in $\sigma$, but fortunately in such a way that we can easily rearrange to find an unbiased estimator)? $\endgroup$ – Silverfish Jan 6 '15 at 0:13
  • 1
    $\begingroup$ @whuber: There should be a similar formula for any location-scale family, with the caveat you pointed out that the function of $n$ may be an intractable integral. $\endgroup$ – Xi'an Jan 6 '15 at 6:15
10
$\begingroup$

Although this is not directly connected to the question, there is a 1968 paper by Peter Bickel and Erich Lehmann that states that, for a convex family of distributions $F$, there exists an unbiased estimator of a functional $q(F)$ (for a sample size $n$ large enough) if and only if $q(\alpha F+(1-\alpha)G)$ is a polynomial in $0\le \alpha\le 1$. This theorem does not apply to the problem here because the collection of Gaussian distributions is not convex (a mixture of Gaussians is not a Gaussian).

An extension of the result in the question is that any power $\sigma^\alpha$ of the standard deviation can be unbiasedly estimated, provided there are enough observations when $\alpha<0$. This follows from the result $$\frac{1}{\sigma^2} \sum_{k=1}^n(x_i-\bar{x})^2 \sim \chi^{2}_{n-1}$$ that $\sigma$ is the scale (and unique) parameter for $\sum_{k=1}^n(x_i-\bar{x})^2$.

This normal setting can then be extended to any location-scale family $$X_1,\ldots,X_n\stackrel{\text{iid}}{\sim} \tau^{-1}f(\tau^{-1}\{x-\mu\})$$ with a finite variance $\sigma^2$. Indeed,

  1. the variance $$\text{var}_{\mu,\tau}(X)=\mathbb{E}_{\mu,\tau}[(X-\mu)^2]=\tau^2\mathbb{E}_{0,1}[X^2]$$ is only a function of $\tau$;
  2. the sum of squares \begin{align*}\mathbb{E}_{\mu,\tau}\left[\sum_{k=1}^n(X_i-\bar{X})^2\right]&=\tau^2\mathbb{E}_{\mu,\tau}\left[\sum_{k=1}^n\tau^{-2}(X_i-\mu-\bar{X}+\mu)^2\right]\\ &=\tau^2\mathbb{E}_{0,1}\left[\sum_{k=1}^n(X_i-\bar{X})^2\right]\end{align*} has an expectation of the form $\tau^2\psi(n)$;
  3. and similarly for any power $$\mathbb{E}_{\mu,\tau}\left[\left\{\sum_{k=1}^n(X_i-\bar{X})^2\right\}^\alpha\right]=\tau^{2\alpha}\mathbb{E}_{0,1}\left[\left\{\sum_{k=1}^n(X_i-\bar{X})^2\right\}^\alpha\right]$$ such that the expectation is finite.
$\endgroup$
10
$\begingroup$

A probably well known case, but a case nevertheless.
Consider a continuous uniform distribution $U(0,\theta)$. Given an i.i.d. sample, the maximum order statistic, $X_{(n)}$ has expected value

$$E(X_{(n)}) = \frac {n}{n+1}\theta $$

The standard deviation of the distribution is

$$\sigma = \frac {\theta}{2\sqrt 3}$$

So the estimator $$\hat \sigma = \frac 1{2\sqrt 3}\frac {n+1}{n}X_{(n)}$$

is evidently unbiased for $\sigma$.

This generalizes to the case where the lower bound of the distribution is also unknown, since we can have an unbiased estimator for the Range, and then the standard deviation is again a linear function of the Range (as is essentially above also).

This exemplifies @whuber's comment, that "the amount of bias is a function of $n$ alone" (plus possibly any known constants) -so it can be deterministically corrected. And this is the case here.

$\endgroup$
  • 4
    $\begingroup$ Now the hard part: when in the world are we interested in the standard deviation of a uniform distribution? (+1) $\endgroup$ – shadowtalker Jan 6 '15 at 16:40
  • 1
    $\begingroup$ @ssdecontrol That's an excellent question! -please proceed to the next one... $\endgroup$ – Alecos Papadopoulos Jan 6 '15 at 17:10
  • 2
    $\begingroup$ One thing I love about this answer is how poor the estimator is. It's quite common to see a question which boils down to "why do we use $\hat{\theta}$ as an estimator even though it's biased?" Some students need convincing that unbiasedness is not the be-all and end-all, and a poor unbiased estimator is one way to show them. $\endgroup$ – Silverfish Jan 9 '15 at 17:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.