2
$\begingroup$

We know that an unbiased estimator of the variance is: $$ \hat{\sigma}^2_{unbiased} = \frac{1}{n-1}\sum_{i=1}^n (x_i - \bar{x})^2$$

I was wondering, does it have the smallest possible standard error?

Does the biased (but consistent) estimator: $$ \hat{\sigma}^2_{biased} = \frac{1}{n}\sum_{i=1}^n (x_i - \bar{x})^2$$ have a lower standard error?

I know that the biased estimator of the variance is the maximum likelihood estimator (MLE) and that MLE estimators have the smallest possible variance in the set of well-behaved estimators. According to this result, what I would conclude here is that $\sigma^2_{biased}$ even though it is biased, has a lower standard error than the unbiased estimator.

$\endgroup$
3
$\begingroup$

Looking at the variance of $\hat{\sigma}_{biased}^2$ we have \begin{eqnarray*} \mathrm{Var}(\hat{\sigma}_{biased}^2) &=& \mathrm{Var} \left( \dfrac{n-1}{n} \hat{\sigma}_{unbiased}^2 \right) \\ &=& \left( \dfrac{n-1}{n} \right)^2 \mathrm{Var}(\hat{\sigma}_{unbiased}^2). \end{eqnarray*} Since $(n-1)/n < 1$ it follows that $$ \mathrm{Var}(\hat{\sigma}_{biased}^2) < \mathrm{Var}(\hat{\sigma}_{unbiased}^2) $$ so $$ \mathrm{sd}(\hat{\sigma}_{biased}^2) < \mathrm{sd}(\hat{\sigma}_{unbiased}^2). $$ The maximum likelihood estimator (MLE) does indeed have a smaller standard error.

$\endgroup$
  • $\begingroup$ So I conclude that among all well behaved estimators, the MLE has the lowest standard error. Also, I was wondering, are they other unbiased variance estimators beside the above formula? If they are, I think it would be interesting to prove that the above formula gives the minimum variance unbiased estimator. $\endgroup$ – Victor Apr 11 at 20:05
  • 1
    $\begingroup$ You can make a variance estimator $ a \hat{\sigma}_{unbiased}^2 $ with arbitrarily low standard error by choosing some $ a < 1 $. For example if $ a = 1/(n + 1) $ you have the minimum mean square error estimator. Smaller values of $ a $ result in more bias so at some point you could say the estimator is no longer well behaved. $\endgroup$ – Estacionario Apr 12 at 5:45
  • 1
    $\begingroup$ I checked the properties of MLE more thoroughly: the MLE has the smallest asymptotic variance. Here, both the unbiased and the biased estimators have the same asymptotic variance, they are both efficient or asymptotically optimal. $\endgroup$ – Victor Apr 14 at 0:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.