2
$\begingroup$

Suppose we know that the correlation between $X_1$ and $Y$ is $\rho_{X_1Y}$ and the correlation between $X_2$ and $Y$ is $\rho_{X_2Y}$. Furthermore, suppose $0 < \rho_{X_2Y} < \rho_{X_1Y}$. Now, we fit the simple linear regression models $Y \sim X_1$ and $Y \sim X_2$, which one will have a lower MSE? i.e. which one has a lower $\parallel Y - \hat{Y} \parallel_2^2 \hspace{0.2cm} = \hspace{0.2cm} \parallel \hat{\epsilon} \parallel_2^2$

I am quite clueless. Is it even possible to say anything about MSE from just the correlation?

$\endgroup$

1 Answer 1

2
$\begingroup$

In simple linear regression, the squared correlation between the $X$ variable and $Y$ variable is equal to the $R^2$.

Also, $R^2$ has an equivalent representation involving MSE:

$$ R^2=1-\dfrac{MSE}{Var(Y)} $$

In both of your simple linear regression equations, the $Y$ is the same, so the $Var(Y)$ in the denominator is the same for each. Consequently, the greater correlation magnitude corresponds to the higher $R^2$ and thus the lower $MSE$, with the extreme being an $MSE$ of zero (correlation magnitude equal to $1$, and $R^2=1$).

This makes sense to me. A high correlation (magnitude) between the predictor and outcome should correspond to a small amount of error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.