Let say you have three variables X1, X2, and Y, all normally distributed, zero mean, unit variance.

When you build a simple linear regression using:

Y ~ X1, the R-Squared is 0.01

Y ~ X2, the R-Squared is 0.02

So my question is: what are the bounds on R-squared when you use both variables? (Y ~ X1 + X2)

My gut instinct is that you can roughly say

  1. The max bound would exist if the features are orthogonal, so 0.03 is the max
  2. The min bound would exist if X2 contains all the information of X1, so 0.02 is the min

Is this correct? What is the mathematical proof for the bounds?

  • 1
    $\begingroup$ If the columns of your data matrix are orthogonal, then the explained variability is the sum of the contribution of each variable. That is almost never the case because variables are often correlated (even weakly), so it will almost certainly be less than the sum of each variable's R^2. $\endgroup$ – Demetri Pananos Feb 14 '19 at 20:35
  • $\begingroup$ @DemetriPananos - as a follow up question, how would you calculate the bounds if you knew the correlation between X1 and X2? $\endgroup$ – user35581 Feb 15 '19 at 14:17
  • $\begingroup$ I think in that situation, the problem is underdetermined. If all I know is the correlation between predictors, I am still missing the variance in the outcome, which is needed to compute R squared. $\endgroup$ – Demetri Pananos Feb 15 '19 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.