3
$\begingroup$

In linear modelling, why do we tend to use the unbiased estimator $\hat{\sigma}^2=\frac{RSS}{n-p}$ instead of the MLE estimator $ \hat{\sigma}^2 = \frac{RSS}{n}$?

I understand it's most likely because it is unbiased, and the MLE estimator is not, but why does this benefit us? Are there general rules for deciding which estimator to use?

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ This Q&A related and possibly relevant. Unbiasedness is probably a big part of the reason. Also, modifications in traditional uses of chi-squared and F-distributions would be necessary before using MLEs because formulas are keyed to the unbiased estimators. $\endgroup$
    – BruceET
    Oct 24, 2020 at 21:25
  • $\begingroup$ I’ll also add that a primary goal of linear regression is inference, not prediction. Those hypothesis tests matter a lot. $\endgroup$
    – Dave
    Oct 24, 2020 at 21:59
  • 2
    $\begingroup$ For the purpose of the usual hypothesis tests, there's no difference whatsoever between the two formulations. (If you were to use the MLE, you would have to adjust the t and F statistics to compensate. On the other hand, if you are using the MLE for $\hat\sigma^2$ then you're probably not even doing t or F tests, but instead are relying on the asymptotic tests for sufficiently large $n,$ indicating there's little difference between $n$ and $n-p$ anyway.) $\endgroup$
    – whuber ♦
    Oct 24, 2020 at 22:05
  • $\begingroup$ One subtle issue here is that at least for estimation in a simple normal setup without regression, the ML $\hat\sigma^2$ with factor $\frac{1}{n}$ has a smaller MSE than the standard one with factor $\frac{1}{n-1}$. I'd think this also holds for regression although I haven't seen a proof (it should be fairly easy to show). Now this seems to favour the ML estimator, however I'd argue that the MSE is somewhat inappropriate for variance estimation, because asymmetry makes underestimation, particularly when getting close to 0, worse than overestimation by the same amount. $\endgroup$
    – Christian Hennig
    Oct 25, 2020 at 10:08
  • 1
    $\begingroup$ I wonder whether anyone has set up an appropriate (how to define that?) asymmetric objective function as alternative to the MSE - the standard estimator with factor $\frac{1}{n-p}$ may look better in this respect, but I haven't seen such a thing. $\endgroup$
    – Christian Hennig
    Oct 25, 2020 at 10:10

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.