# Possible range of $R^2$

Suppose are three time series, $$X_1$$, $$X_2$$ and $$Y$$

Running ordinary linear regression on $$Y$$ ~ $$X_1$$ ($$Y = b X_1 + b_0 + \epsilon$$ ), we get $$R^2 = U$$. The ordinary linear regression $$Y$$ ~ $$X_2$$ get $$R^2 = V$$. Assume $$U < V$$

What's the minimum and maximum possible values of $$R^2$$ on regression $$Y$$ ~ $$X_1 + X_2$$ ($$Y = b_1 X_1 + b_2 X_2 + b_0 + \epsilon$$ )?

I believe the minimum $$R^2$$ should be $$V$$ + a small value, since adding new variables always increases $$R^2$$, but I don't know how to quantify this small value, and I don't know how to obtain the maximum range.

1) EDIT: Cardinal's comment below shows that the correct answer to the min $R^2$ question is $V$. Hence I'm deleting my "interesting", but ultimately incorrect, answer to that part of the OP's post.

2) The maximum $R^2$ is 1. Consider the following example, which fits your case.

x1 <- rnorm(100)
x2 <- rnorm(100)
y <- x1 + 2*x2

> summary(lm(y~x1))$r.squared  0.2378023 # This is U > summary(lm(y~x2))$r.squared
 0.7917808                 # This is V; U < V
> summary(lm(y~x1+x2))$r.squared  1  Here we are fixing the variance of$\epsilon$at 0. If you want$\sigma^2_\epsilon > 0$, though, things change a little. You can get the$R^2$arbitrarily close to 1 by making$\sigma^2_\epsilon$smaller and smaller, but, as with the minimum problem, you can't get there, so there is no maximum. 1 becomes the supremum, since it's always greater than$R^2$but it's also the limit as$\sigma^2_\epsilon \to 0$. • (+1) Some comments: This is a good answer; it's interesting that you've taken an asymptotic approach whereas it's not clear whether the OP was interested in that or, possible, a fixed-$n$one (or both). This answer is a little inconsistent with the OP's constraint that$U < V$, though, and if$X_1 = 0$or$X_1 = a \mathbf{1}$for some$a \in \mathbb R$, for example, then the minimum$R^2$for all fixed sample sizes is exactly$V := V(n)$. (Excuse the pathology of these examples.) Also, OLS is not necessarily consistent absent additional constraints on the predictors. :) Jul 14, 2012 at 15:28 • @cardinal - on rereading, I can't figure out why I took that approach to the min problem, when$V$now seems like the obviously correct answer and, as you've implicitly observed, I could have constructed an example that achieves it in the vein of the max part... oh well, maybe my espresso this morning was accidentally decaf. (Maybe I should review my answers more thoroughly before posting, too!) Jul 15, 2012 at 2:24 • I don't think you should remove what you've written, which I did find an interesting approach to answering the question! While the pathologies I mention certainly allow for a minimum$R^2$, one might wonder what is really meant by$X_1 = 0$. The other example is perhaps not quite as pathological since in a general version of this problem, it extends to the case where any additional$X_i$is in the column space of the other predictors. :) Jul 15, 2012 at 2:31 • @cardinal - thanks! I'll reconstruct it, maybe a little more formally, and put it back in at the bottom in a while. Jul 15, 2012 at 2:35 With no constraints on$U$and$V$, then the minimum is$V$, and then maximum is the smaller$\min(V + U, 1)$. This is because two variable could be perfectly correlated (in which case adding the second variable does not change the$R^2$at all) or they could be orthogonal in which case including both results in$U + V$. It was rightly pointed out in the comments that this also requires that each be orthogonal to$\mathbf{1}$, the column vector of 1s. You added the constraint$U < V \implies X_{1} \neq X_{2}$. However, it is still possible that$U = 0$. That is,$X_{1} \perp Y$, in which case,$\min = \max = V + 0$. Finally, it is possible that$X_{1} \perp X_{2}$so the upper bound is still$\min(V + U, 1)$. If you knew more about the relationship between$X_{1}$and$X_{2}$, I think you could say more. • (+1) But, note that it is not (quite) true that if$X_1$and$X_2$are orthogonal, then their individual$R^2$values will sum when including both in the model. We also need them to be orthogonal to the all-ones vector$\mathbf 1$. Note that you can use$\LaTeX$on this site for marking up the math. :) Jul 15, 2012 at 22:00 • That is true. Thanks very much for the comments, and for pointing out that$\LaTeX$can be used. I thought it might but had tried mathjax style escapes ( and [ for inline/equations. Writing just like I would in TeX worked like a charm :) Jul 16, 2012 at 3:35 • From @Magot's answer below, what happens if$r_{1,Y}$is negative, and$r_{1,2}$is non-zero? We can find some setting here when$R^2$is greater than$U+V$and close to$1$, but$U$and$V$are less than say$0.3$each. Oct 27, 2021 at 4:04 Let$r_{1,2}$equal the correlation between$X_1$and$X_2$,$r_{1,Y}$equal the correlation between$X_1$and$Y$, and$r_{2,Y}$the correlation between$X_2$and$Y$. Then$R^2$for the full model divided by$V$equals $$\left(\frac{1}{(1 - r_{1,2}^2)}\right) \left(1 - \frac{2 \cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} + \frac{U}{V}\right).$$ So$R^2$for the full model equals$V$only if$r_{1,2} = 0$and$r_{1,Y}^2 = U = 0$or $$r_{1,2}^2 = \frac{2\cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} - \frac{U}{V}.$$ If$r_{1,2} = 0$,$R^2$for the full model equals$U + V\$.

• (+1) Cute. Welcome to the site. Please consider registering your account so you can participate more fully. I'll have to look at this expression a little more closely later on. :) Jul 18, 2012 at 12:54