2
$\begingroup$

For OLS linear regression, Hayes (2022, p. 56) provides a definition of "mean squared residual" and one for "standard error of estimate", for a model with $k$ predictor variables:

$$\large \text{mean squared residual}=\frac{\sum^n_{i=1} (y_i-\hat y_i)^2}{n-(k+1)}=\frac{\sum^n_{i=1} \varepsilon_i^2}{n-(k+1)}$$ $$\large \text{standard error of estimate}=\sqrt\frac{\sum^n_{i=1} \varepsilon_i^2}{n-(k+1)}$$

However, the formula for $R^2$ implies there is another type of "mean squared residual" with only $(n-1)$ in the denominator: $$\large R^2=\frac{\hat\sigma^2_{\hat{Y}}}{\hat\sigma^2_Y}=\frac{\hat\sigma^2_Y-\hat\sigma^2_\varepsilon}{\hat\sigma^2_Y}=\frac{\frac{\sum^n_{i=1} (y_i-\bar Y)^2}{(n-1)} -\frac{\sum^n_{i=1} \varepsilon^2}{(n-1)}}{\frac{\sum^n_{i=1} (y_i-\bar Y)^2}{(n-1)}}$$

Thus, what would be the name for $\large\frac{\sum^n_{i=1} \varepsilon^2}{(n-1)}$ to distinguish it from $\large\frac{\sum^n_{i=1} \varepsilon_i^2}{n-(k+1)}$ ?

References

$\endgroup$
3
  • 1
    $\begingroup$ Side remark: using the $k-1$ version to calculate R-squared gives another good friend: the R-squared adjusted. $\endgroup$
    – Michael M
    Oct 14, 2023 at 19:27
  • $\begingroup$ There isn't a question here, because there is no "$n-1$" in the denominator since all those factors cancel: you could replace them with any nonzero number you please. $\endgroup$
    – whuber
    Oct 14, 2023 at 21:09
  • $\begingroup$ @whuber I know they cancel out. For R^2, would it be legitimate to make all the denominators (n-k-1)? If so, I would think those terms would have different names than their (n-1) counterparts, and that’s my question: what is the name of the squared error with df (n-1), versus its name with df (n-k-1). I assume they must have different names. $\endgroup$ Oct 15, 2023 at 2:05

1 Answer 1

-1
$\begingroup$

Okay, I think I may have an answer, but I'm not 100% sure of its correctness. Perhaps: $$\large\displaystyle\frac{\sum^n_{i=1}(y_i-\hat y_i)^2}{n-1}=\small\text{unbiased estimate of the {population mean squared residuals}}$$ $$\large\displaystyle \sqrt\frac{\sum^n_{i=1}(y_i-\hat y_i)^2}{n-(k+1)}=\small\text{unbiased estimate of the {population standard error of residuals}}$$

The above is based on analogical reasoning from the following:

$\large\displaystyle\frac{\sum^n_{i=1}(y_i-\bar Y)^2}{n-1}$ is the unbiased estimate of the population variance of $Y$ , and $\large\displaystyle \sqrt \frac{\sum^n_{i=1}(y_i-\bar Y)^2}{n-1}$ is a biased estimate of the population standard deviation of $Y$, hence my proposed answer above. If someone could help confirm or challenge the correctness of this, I would feel better.

$\endgroup$
1
  • $\begingroup$ I suspect my answer is incorrect: I think the top formula in my answer is merely {the unbiased estimate of the variance of the residuals}, whereas the second formula in my answer is the {root mean squared error (RMSE)} (which is merely the square root of the {mean squared error (MSE)}). As far as I understand, they each have a purpose: the unbiased estimate of the variance of the residuals is focused on accurately estimating the variance of the residuals, whereas the MSE and RMSE are focused on evaluating the predictive performance of the model while considering its degrees of freedom. $\endgroup$ Dec 29, 2023 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.