Possible range of $R^2$

Question

Suppose are three time series, $X_1$, $X_2$ and $Y$

Running ordinary linear regression on $Y$ ~ $X_1$ ($Y = b X_1 + b_0 + \epsilon$ ), we get $R^2 = U$. The ordinary linear regression $Y$ ~ $X_2$ get $R^2 = V$. Assume $U < V$

What's the minimum and maximum possible values of $R^2$ on regression $Y$ ~ $X_1 + X_2$ ($Y = b_1 X_1 + b_2 X_2 + b_0 + \epsilon$ )?

I believe the minimum $R^2$ should be $V$ + a small value, since adding new variables always increases $R^2$, but I don't know how to quantify this small value, and I don't know how to obtain the maximum range.

Thanks a lot!

Answer 1

1) EDIT: Cardinal's comment below shows that the correct answer to the min $R^2$ question is $V$. Hence I'm deleting my "interesting", but ultimately incorrect, answer to that part of the OP's post.

2) The maximum $R^2$ is 1. Consider the following example, which fits your case.

x1 <- rnorm(100)
x2 <- rnorm(100)
y <- x1 + 2*x2

> summary(lm(y~x1))$r.squared
[1] 0.2378023                 # This is U
> summary(lm(y~x2))$r.squared
[1] 0.7917808                 # This is V; U < V
> summary(lm(y~x1+x2))$r.squared
[1] 1

Here we are fixing the variance of $\epsilon$ at 0. If you want $\sigma^2_\epsilon > 0$, though, things change a little. You can get the $R^2$ arbitrarily close to 1 by making $\sigma^2_\epsilon$ smaller and smaller, but, as with the minimum problem, you can't get there, so there is no maximum. 1 becomes the supremum, since it's always greater than $R^2$ but it's also the limit as $\sigma^2_\epsilon \to 0$.

Answer 2

Let $r_{1,2}$ equal the correlation between $X_1$ and $X_2$, $r_{1,Y}$ equal the correlation between $X_1$ and $Y$, and $r_{2,Y}$ the correlation between $X_2$ and $Y$. Then $R^2$ for the full model divided by $V$ equals

$$\left(\frac{1}{(1 - r_{1,2}^2)}\right) \left(1 - \frac{2 \cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} + \frac{U}{V}\right).$$

So $R^2$ for the full model equals $V$ only if $r_{1,2} = 0$ and $r_{1,Y}^2 = U = 0$ or

$$r_{1,2}^2 = \frac{2\cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} - \frac{U}{V}.$$

If $r_{1,2} = 0$, $R^2$ for the full model equals $U + V$.

Answer 3

With no constraints on $U$ and $V$, then the minimum is $V$, and then maximum is the smaller $\min(V + U, 1)$. This is because two variable could be perfectly correlated (in which case adding the second variable does not change the $R^2$ at all) or they could be orthogonal in which case including both results in $U + V$. It was rightly pointed out in the comments that this also requires that each be orthogonal to $\mathbf{1}$, the column vector of 1s.

You added the constraint $U < V \implies X_{1} \neq X_{2}$. However, it is still possible that $U = 0$. That is, $X_{1} \perp Y$, in which case, $\min = \max = V + 0$. Finally, it is possible that $X_{1} \perp X_{2}$ so the upper bound is still $\min(V + U, 1)$.

If you knew more about the relationship between $X_{1}$ and $X_{2}$, I think you could say more.