# What's the difference between "standard error of estimate" versus the "variance of residuals"

For OLS linear regression, Hayes (2022, p. 56) provides a definition of "mean squared residual" and one for "standard error of estimate", for a model with $$k$$ predictor variables:

$$\large \text{mean squared residual}=\frac{\sum^n_{i=1} (y_i-\hat y_i)^2}{n-(k+1)}=\frac{\sum^n_{i=1} \varepsilon_i^2}{n-(k+1)}$$ $$\large \text{standard error of estimate}=\sqrt\frac{\sum^n_{i=1} \varepsilon_i^2}{n-(k+1)}$$

However, the formula for $$R^2$$ implies there is another type of "mean squared residual" with only $$(n-1)$$ in the denominator: $$\large R^2=\frac{\hat\sigma^2_{\hat{Y}}}{\hat\sigma^2_Y}=\frac{\hat\sigma^2_Y-\hat\sigma^2_\varepsilon}{\hat\sigma^2_Y}=\frac{\frac{\sum^n_{i=1} (y_i-\bar Y)^2}{(n-1)} -\frac{\sum^n_{i=1} \varepsilon^2}{(n-1)}}{\frac{\sum^n_{i=1} (y_i-\bar Y)^2}{(n-1)}}$$

Thus, what would be the name for $$\large\frac{\sum^n_{i=1} \varepsilon^2}{(n-1)}$$ to distinguish it from $$\large\frac{\sum^n_{i=1} \varepsilon_i^2}{n-(k+1)}$$ ?

References

• Side remark: using the $k-1$ version to calculate R-squared gives another good friend: the R-squared adjusted. Oct 14, 2023 at 19:27
• There isn't a question here, because there is no "$n-1$" in the denominator since all those factors cancel: you could replace them with any nonzero number you please.
– whuber
Oct 14, 2023 at 21:09
• @whuber I know they cancel out. For R^2, would it be legitimate to make all the denominators (n-k-1)? If so, I would think those terms would have different names than their (n-1) counterparts, and that’s my question: what is the name of the squared error with df (n-1), versus its name with df (n-k-1). I assume they must have different names. Oct 15, 2023 at 2:05

Okay, I think I may have an answer, but I'm not 100% sure of its correctness. Perhaps: $$\large\displaystyle\frac{\sum^n_{i=1}(y_i-\hat y_i)^2}{n-1}=\small\text{unbiased estimate of the {population mean squared residuals}}$$ $$\large\displaystyle \sqrt\frac{\sum^n_{i=1}(y_i-\hat y_i)^2}{n-(k+1)}=\small\text{unbiased estimate of the {population standard error of residuals}}$$
$$\large\displaystyle\frac{\sum^n_{i=1}(y_i-\bar Y)^2}{n-1}$$ is the unbiased estimate of the population variance of $$Y$$ , and $$\large\displaystyle \sqrt \frac{\sum^n_{i=1}(y_i-\bar Y)^2}{n-1}$$ is a biased estimate of the population standard deviation of $$Y$$, hence my proposed answer above. If someone could help confirm or challenge the correctness of this, I would feel better.