2
$\begingroup$

Assume we have two variables $X_1$ and $X_2$. If I do a linear regression of $Y$ on $X_1$, I have $R_1^2$. Similarly, I get $R_2^2$ for regression of $Y$ on $X_2$. If I run a regression of $Y$ on $X_1$ and $X_2$ together and assume that $R_1^2+R_2^2 < 1$, would it be possible to get a total $R^2$ larger than $R_1^2+R_2^2$ ?

My gut feeling is impossible, $R_1^2+R_2^2$ must be the upper bound since this is the maximum acquired when $X_1$ and $X_2$ are independent.

I have seen posts here and here. However, I'm still not sure whether we can get a range for the $\rho_{12}$, which is the correlation between those 2 variables, from $R_1^2$ and $R_2^2$, so that at least we can use that to analyze the range for the total $R^2$ below: $$R^2 = \frac{R_1^2 + R_2^2 - 2R_1R_2\rho_{12}}{1-\rho^2_{12}}$$.

Also, it would be fantastic if someone could point out a more intuitive or geometry explanation. Thanks.

$\endgroup$
1
$\begingroup$

Yes, it is possible. If both $X_1$ and $X_2$ are positively correlated with $Y$ but negatively correlated with each other.

$\endgroup$
  • $\begingroup$ Thanks. Could you give some intuition why? $\endgroup$ – James May 14 '18 at 23:29
  • 1
    $\begingroup$ See this $\endgroup$ – dbwilson May 15 '18 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.