3
$\begingroup$

Assume we have two variables $X_1$ and $X_2$. If I do a linear regression of $Y$ on $X_1$, I have $R_1^2$. Similarly, I get $R_2^2$ for regression of $Y$ on $X_2$. If I run a regression of $Y$ on $X_1$ and $X_2$ together and assume that $R_1^2+R_2^2 < 1$, would it be possible to get a total $R^2$ larger than $R_1^2+R_2^2$ ?

My gut feeling is impossible, $R_1^2+R_2^2$ must be the upper bound since this is the maximum acquired when $X_1$ and $X_2$ are independent.

I have seen posts here and here. However, I'm still not sure whether we can get a range for the $\rho_{12}$, which is the correlation between those 2 variables, from $R_1^2$ and $R_2^2$, so that at least we can use that to analyze the range for the total $R^2$ below: $$R^2 = \frac{R_1^2 + R_2^2 - 2R_1R_2\rho_{12}}{1-\rho^2_{12}}$$.

Also, it would be fantastic if someone could point out a more intuitive or geometry explanation. Thanks.

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, it is possible. If both $X_1$ and $X_2$ are positively correlated with $Y$ but negatively correlated with each other.

$\endgroup$
2
  • $\begingroup$ Thanks. Could you give some intuition why? $\endgroup$
    – James LT
    May 14, 2018 at 23:29
  • 1
    $\begingroup$ See this $\endgroup$
    – dbwilson
    May 15, 2018 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.