6
$\begingroup$

We know that an unbiased estimator of the variance is: $$ \hat{\sigma}^2_{unbiased} = \frac{1}{n-1}\sum_{i=1}^n (x_i - \bar{x})^2$$

I was wondering, does it have the smallest possible standard error?

Does the biased (but consistent) estimator: $$ \hat{\sigma}^2_{biased} = \frac{1}{n}\sum_{i=1}^n (x_i - \bar{x})^2$$ have a lower standard error?

I know that the biased estimator of the variance is the maximum likelihood estimator (MLE) and that MLE estimators have the smallest possible variance in the set of well-behaved estimators. According to this result, what I would conclude here is that $\sigma^2_{biased}$ even though it is biased, has a lower standard error than the unbiased estimator.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

Looking at the variance of $\hat{\sigma}_{biased}^2$ we have \begin{eqnarray*} \mathrm{Var}(\hat{\sigma}_{biased}^2) &=& \mathrm{Var} \left( \dfrac{n-1}{n} \hat{\sigma}_{unbiased}^2 \right) \\ &=& \left( \dfrac{n-1}{n} \right)^2 \mathrm{Var}(\hat{\sigma}_{unbiased}^2). \end{eqnarray*} Since $(n-1)/n < 1$ it follows that $$ \mathrm{Var}(\hat{\sigma}_{biased}^2) < \mathrm{Var}(\hat{\sigma}_{unbiased}^2) $$ so $$ \mathrm{sd}(\hat{\sigma}_{biased}^2) < \mathrm{sd}(\hat{\sigma}_{unbiased}^2). $$ The maximum likelihood estimator (MLE) does indeed have a smaller standard error.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So I conclude that among all well behaved estimators, the MLE has the lowest standard error. Also, I was wondering, are they other unbiased variance estimators beside the above formula? If they are, I think it would be interesting to prove that the above formula gives the minimum variance unbiased estimator. $\endgroup$
    – Victor
    Apr 11, 2019 at 20:05
  • 1
    $\begingroup$ You can make a variance estimator $ a \hat{\sigma}_{unbiased}^2 $ with arbitrarily low standard error by choosing some $ a < 1 $. For example if $ a = 1/(n + 1) $ you have the minimum mean square error estimator. Smaller values of $ a $ result in more bias so at some point you could say the estimator is no longer well behaved. $\endgroup$
    – Estacionario
    Apr 12, 2019 at 5:45
  • 1
    $\begingroup$ I checked the properties of MLE more thoroughly: the MLE has the smallest asymptotic variance. Here, both the unbiased and the biased estimators have the same asymptotic variance, they are both efficient or asymptotically optimal. $\endgroup$
    – Victor
    Apr 14, 2019 at 0:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.