3
$\begingroup$

The bias-corrected sample variance $$ \hat{\sigma}^2 = \dfrac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2 $$ is commonly used in natural sciences to estimate the variance $ \sigma^2 $ of a Gaussian random variable $ X $ from an i.i.d. sample $ X_1, \dots, X_n $.

One of the earliest things I learned in statistical inference classes is that bias is only one of many relevant properties when comparing estimators. My understanding is that mean square error (MSE) is generally a better measure of estimator performance than bias alone.

Why then is the minimum MSE estimator $$ \hat{\sigma}_{MMSE}^2 = \dfrac{1}{n+1} \sum_{i=1}^n (X_i - \bar{X})^2 $$ not the standard choice for this problem?

Improve this question
$\endgroup$
1
  • $\begingroup$ Obviously, partly because unbiasedness is preferred over minmizing MSE. $\endgroup$ – BruceET Apr 11 '19 at 23:50
1
$\begingroup$

Calculating sample variance as squared deviation divided by n + 1 (instead of n - 1) will lead to underestimating variance, which will lead to confidence intervals with lower than expected probability coverage. That's probably at least part of the reason that an unbiased estimator of variance is preferred over minimizing MSE.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Actually, one could perfectly well use the MMSE estimator in formulas for confidence intervals--but that would merely complicate them, because factors of $\sqrt{(n+1)/(n-1)}$ would need to be introduced! $\endgroup$ – whuber♦ Nov 23 '19 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.