0
$\begingroup$

Is the root mean square error (RMSE) of a regression related to the standard error of its regression coefficients? If so, how?

$\endgroup$
1
$\begingroup$

Under the classical setting of the multiple regression, we know that $$ E||y-\hat{y}||_2^2 = {\rm tr}\big[\sigma^2 (I - X(X^{\rm T} X)^{-1} X^{\rm T})\big] = \sigma^2 (n - {\rm rank}(X)), $$ where the last equality holds from a trace of an idempotent matrix. Also, $$ {\rm Var}(\hat{\beta}) = \sigma^2 (X^{\rm T} X)^{-1}. $$ From the facts above, it can be said that if covariates are more correlated, then (1) the rank of $X$ is small so that the RMSE gets larger, and (2) $\hat{\beta}$ would have more variation from multicolinearity.

Does this make sense to you?

$\endgroup$
1
  • $\begingroup$ The rank of X will not be lowered by multicollinearity unless the multicollinearity is perfect, although the condition number of X will be adversely affected. $\endgroup$ – EdM Feb 14 '20 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.