# Possible range of $R^2$

Suppose are three time series, $X_1$, $X_2$ and $Y$

Running ordinary linear regression on $Y$ ~ $X_1$ ($Y = b X_1 + b_0 + \epsilon$ ), we get $R^2 = U$. The ordinary linear regression $Y$ ~ $X_2$ get $R^2 = V$. Assume $U < V$

What's the minimum and maximum possible values of $R^2$ on regression $Y$ ~ $X_1 + X_2$ ($Y = b_1 X_1 + b_2 X_2 + b_0 + \epsilon$ )?

I believe the minimum $R^2$ should be $V$ + a small value, since adding new variables always increases $R^2$, but I don't know how to quantify this small value, and I don't know how to obtain the maximum range.

Thanks a lot!

-

1) EDIT: Cardinal's comment below shows that the correct answer to the min $R^2$ question is $V$. Hence I'm deleting my "interesting", but ultimately incorrect, answer to that part of the OP's post.
2) The maximum $R^2$ is 1. Consider the following example, which fits your case.
x1 <- rnorm(100)
> summary(lm(y~x1))$r.squared [1] 0.2378023 # This is U > summary(lm(y~x2))$r.squared
> summary(lm(y~x1+x2))$r.squared [1] 1  Here we are fixing the variance of$\epsilon$at 0. If you want$\sigma^2_\epsilon > 0$, though, things change a little. You can get the$R^2$arbitrarily close to 1 by making$\sigma^2_\epsilon$smaller and smaller, but, as with the minimum problem, you can't get there, so there is no maximum. 1 becomes the supremum, since it's always greater than$R^2$but it's also the limit as$\sigma^2_\epsilon \to 0$. - (+1) Some comments: This is a good answer; it's interesting that you've taken an asymptotic approach whereas it's not clear whether the OP was interested in that or, possible, a fixed-$n$one (or both). This answer is a little inconsistent with the OP's constraint that$U < V$, though, and if$X_1 = 0$or$X_1 = a \mathbf{1}$for some$a \in \mathbb R$, for example, then the minimum$R^2$for all fixed sample sizes is exactly$V := V(n)$. (Excuse the pathology of these examples.) Also, OLS is not necessarily consistent absent additional constraints on the predictors. :) – cardinal Jul 14 '12 at 15:28 @cardinal - on rereading, I can't figure out why I took that approach to the min problem, when$V$now seems like the obviously correct answer and, as you've implicitly observed, I could have constructed an example that achieves it in the vein of the max part... oh well, maybe my espresso this morning was accidentally decaf. (Maybe I should review my answers more thoroughly before posting, too!) – jbowman Jul 15 '12 at 2:24 I don't think you should remove what you've written, which I did find an interesting approach to answering the question! While the pathologies I mention certainly allow for a minimum$R^2$, one might wonder what is really meant by$X_1 = 0$. The other example is perhaps not quite as pathological since in a general version of this problem, it extends to the case where any additional$X_i$is in the column space of the other predictors. :) – cardinal Jul 15 '12 at 2:31 @cardinal - thanks! I'll reconstruct it, maybe a little more formally, and put it back in at the bottom in a while. – jbowman Jul 15 '12 at 2:35 add comment Let$r_{1,2}$equal the correlation between$X_1$and$X_2$,$r_{1,Y}$equal the correlation between$X_1$and$Y$, and$r_{2,Y}$the correlation between$X_2$and$Y$. Then$R^2$for the full model divided by$V$equals $$\left(\frac{1}{(1 - r_{1,2}^2)}\right) \left(1 - \frac{2 \cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} + \frac{U}{V}\right).$$ So$R^2$for the full model equals$V$only if$r_{1,2} = 0$and$r_{1,Y}^2 = U = 0$or $$r_{1,2}^2 = \frac{2\cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} - \frac{U}{V}.$$ If$r_{1,2} = 0$,$R^2$for the full model equals$U + V$. - (+1) Cute. Welcome to the site. Please consider registering your account so you can participate more fully. I'll have to look at this expression a little more closely later on. :) – cardinal Jul 18 '12 at 12:54 add comment With no constraints on$U$and$V$, then the minimum is$V$, and then maximum is the smaller$\min(V + U, 1)$. This is because two variable could be perfectly correlated (in which case adding the second variable does not change the$R^2$at all) or they could be orthogonal in which case including both results in$U + V$. It was rightly pointed out in the comments that this also requires that each be orthogonal to$\mathbf{1}$, the column vector of 1s. You added the constraint$U < V \implies X_{1} \neq X_{2}$. However, it is still possible that$U = 0$. That is,$X_{1} \perp Y$, in which case,$\min = \max = V + 0$. Finally, it is possible that$X_{1} \perp X_{2}$so the upper bound is still$\min(V + U, 1)$. If you knew more about the relationship between$X_{1}$and$X_{2}$, I think you could say more. - (+1) But, note that it is not (quite) true that if$X_1$and$X_2$are orthogonal, then their individual$R^2$values will sum when including both in the model. We also need them to be orthogonal to the all-ones vector$\mathbf 1$. Note that you can use$\LaTeX$on this site for marking up the math. :) – cardinal Jul 15 '12 at 22:00 That is true. Thanks very much for the comments, and for pointing out that$\LaTeX\$ can be used. I thought it might but had tried mathjax style escapes ( and [ for inline/equations. Writing just like I would in TeX worked like a charm :) –  Joshua Jul 16 '12 at 3:35