Assuming the true equation for Y is linear as below: $$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$ Assuming X is fixed, then the variance of each Y is: $$var(Y_i )=var(\epsilon_i)=\sigma^2$$ In order to estimate $\sigma ^2$, we usually use the mse: $$\hat{\sigma}^2=\frac{1}{n-p-1}\sum_i(Y_i -\hat{Y_i})^2 $$ My question is, why can't we estimate $\sigma^2$ using the sample variance below? $$\hat{\sigma}^2=\frac{1}{n-1}\sum_i (Y_i-\bar{Y})^2$$ The sample variance is also an unbiased estimator, so instead of predicted value we just use the sample average of Y?

  • 1
    $\begingroup$ You say "Assuming $X$ is fixed" but, random or not, the values of $X_i$ usually differ (if they are all the same then linear regression does not work) $\endgroup$
    – Henry
    Commented Aug 12, 2022 at 22:00
  • $\begingroup$ yes, what I mean is the data matrix of X is fixed, each Xi can differ $\endgroup$
    – woowz
    Commented Aug 12, 2022 at 22:05
  • $\begingroup$ The second display actually isn't right. You mean to write: $\text{var}(Y_i | X) = \text{var}(\epsilon) = \sigma^2$. $\endgroup$
    – AdamO
    Commented Aug 12, 2022 at 22:24
  • $\begingroup$ If X is fixed, conditioning on it does not make a difference $\endgroup$
    – woowz
    Commented Aug 12, 2022 at 22:25

1 Answer 1


$\hat\sigma=\frac{1}{N-1}\sum_{i=1}^N\left(Y_i-\bar Y\right)^2$ has nothing to do with the model, unless your model is to predict the mean $\bar Y$ every single time, no matter what the features are.

A measure of prediction quality that is unrelated to the predictions does not make much sense.

In fact, the reason for doing regression is to tighten up your estimates and do a better job of predicting the conditional mean than you would do by predicting the pooled/marginal mean every time.

Your $\hat\sigma$ does come up in $R^2$, however, as $R^2=1-\dfrac{ \sum_{i=1}^N\left( Y_i -\hat Y_i \right)^2 }{ (N-1)\frac{1}{N-1}\sum_{i=1}^N\left(Y_i-\bar Y\right)^2 }=1-\dfrac{ \sum_{i=1}^N\left( Y_i -\hat Y_i \right)^2 }{ (N-1)\hat\sigma }$.


Here is a quick simulation to show that the two equations give dramatically different results, even for a "large" sample size of a million.

N <- 1000000
x <- runif(N, 0, 12)
y <- x + rnorm(N)
L <- lm(y ~ x)
(1 / (N - 2)) * sum((y - predict(L))^2) # I get 0.9992724.
(1 / (N - 1)) * sum((y - mean(y))^2) # I get 12.98154.


"Sigma" has two meanings. You are right that $\sigma^2$ can be used to denote the constant variance of the errors. This would related to the conditional distribution, which is our object of interest in regression, and $\frac{1}{n-p-1}\sum_i(Y_i -\hat{Y_i})^2 $ is unbiased for this conditional variance.

However, $\sigma^2$ also could apply just to $Y$, marginally/pooled, in which case, $\frac{1}{n-1}\sum_i(Y_i -\bar{Y})^2 $ is an unbiased estimator for this unconditional/marginal/pooled variance.

If you want to differentiate between them, perhaps consider using subscripts like $\sigma^2_{Y\vert X}$ for the conditional variance and $\sigma^2_{Y}$ for the unconditional variance, but they refer to different random variables that have different variances.

  • $\begingroup$ I understand using the average has nothing to do with the model. But as u can see, both are unbiased/consistent. So it does not matter which one you use assuming you have a large sample size Are you saying that if the relationship is truly linear and we have small samples, using the MSE version will be more accurate than if we used the sample variance. if we have large samples does this really matter? $\endgroup$
    – woowz
    Commented Aug 12, 2022 at 22:23
  • $\begingroup$ @woowz Unbiased for what? The "divide by N" variance formula is biased for (marginal) population variance, leading to Bessel's correction and the "N-1" denominator in the usual "sample variance", $S^2$. $\endgroup$
    – Dave
    Commented Aug 12, 2022 at 22:28
  • $\begingroup$ Oh yes my bad, i've edited to include N-1. So now both are unbiased estimators for $\sigma^2$, and so there is really no difference in using either one in large sample sizes. Do you agree with this statement? $\endgroup$
    – woowz
    Commented Aug 12, 2022 at 22:31
  • $\begingroup$ The fractions will be about the same in large sample sizes, as "huge minus one" is almost the same as "huge minus a few", but there is an enormous difference between subtracting the marginal mean and subtracting the predictions, unless you are evaluating a model that always predicts the marginal mean. $\endgroup$
    – Dave
    Commented Aug 12, 2022 at 22:34
  • 1
    $\begingroup$ oh yes, you are correct! As you mentioned, if you treat regressors as "fixed", then the distinction between unconditional and conditional variance gets lost, which resulted in the incorrect derivation of my second equation! MInor edit: In your second paragraph I believe it should be Y bar instead of Y hat for the unconditional variance. $\endgroup$
    – woowz
    Commented Aug 12, 2022 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.