Suppose are three time series, $X_1$, $X_2$ and $Y$

Running ordinary linear regression on $Y$ ~ $X_1$ ($Y = b X_1 + b_0 + \epsilon$ ), we get $R^2 = U$. The ordinary linear regression $Y$ ~ $X_2$ get $R^2 = V$. Assume $U < V$

What's the minimum and maximum possible values of $R^2$ on regression $Y$ ~ $X_1 + X_2$ ($Y = b_1 X_1 + b_2 X_2 + b_0 + \epsilon$ )?

I believe the minimum $R^2$ should be $V$ + a small value, since adding new variables always increases $R^2$, but I don't know how to quantify this small value, and I don't know how to obtain the maximum range.


3 Answers 3


1) EDIT: Cardinal's comment below shows that the correct answer to the min $R^2$ question is $V$. Hence I'm deleting my "interesting", but ultimately incorrect, answer to that part of the OP's post.

2) The maximum $R^2$ is 1. Consider the following example, which fits your case.

x1 <- rnorm(100)
x2 <- rnorm(100)
y <- x1 + 2*x2

> summary(lm(y~x1))$r.squared
[1] 0.2378023                 # This is U
> summary(lm(y~x2))$r.squared
[1] 0.7917808                 # This is V; U < V
> summary(lm(y~x1+x2))$r.squared
[1] 1

Here we are fixing the variance of $\epsilon$ at 0. If you want $\sigma^2_\epsilon > 0$, though, things change a little. You can get the $R^2$ arbitrarily close to 1 by making $\sigma^2_\epsilon$ smaller and smaller, but, as with the minimum problem, you can't get there, so there is no maximum. 1 becomes the supremum, since it's always greater than $R^2$ but it's also the limit as $\sigma^2_\epsilon \to 0$.

  • 2
    $\begingroup$ (+1) Some comments: This is a good answer; it's interesting that you've taken an asymptotic approach whereas it's not clear whether the OP was interested in that or, possible, a fixed-$n$ one (or both). This answer is a little inconsistent with the OP's constraint that $U < V$, though, and if $X_1 = 0$ or $X_1 = a \mathbf{1}$ for some $a \in \mathbb R$, for example, then the minimum $R^2$ for all fixed sample sizes is exactly $V := V(n)$. (Excuse the pathology of these examples.) Also, OLS is not necessarily consistent absent additional constraints on the predictors. :) $\endgroup$
    – cardinal
    Jul 14, 2012 at 15:28
  • $\begingroup$ @cardinal - on rereading, I can't figure out why I took that approach to the min problem, when $V$ now seems like the obviously correct answer and, as you've implicitly observed, I could have constructed an example that achieves it in the vein of the max part... oh well, maybe my espresso this morning was accidentally decaf. (Maybe I should review my answers more thoroughly before posting, too!) $\endgroup$
    – jbowman
    Jul 15, 2012 at 2:24
  • $\begingroup$ I don't think you should remove what you've written, which I did find an interesting approach to answering the question! While the pathologies I mention certainly allow for a minimum $R^2$, one might wonder what is really meant by $X_1 = 0$. The other example is perhaps not quite as pathological since in a general version of this problem, it extends to the case where any additional $X_i$ is in the column space of the other predictors. :) $\endgroup$
    – cardinal
    Jul 15, 2012 at 2:31
  • 1
    $\begingroup$ @cardinal - thanks! I'll reconstruct it, maybe a little more formally, and put it back in at the bottom in a while. $\endgroup$
    – jbowman
    Jul 15, 2012 at 2:35

With no constraints on $U$ and $V$, then the minimum is $V$, and then maximum is the smaller $\min(V + U, 1)$. This is because two variable could be perfectly correlated (in which case adding the second variable does not change the $R^2$ at all) or they could be orthogonal in which case including both results in $U + V$. It was rightly pointed out in the comments that this also requires that each be orthogonal to $\mathbf{1}$, the column vector of 1s.

You added the constraint $U < V \implies X_{1} \neq X_{2}$. However, it is still possible that $U = 0$. That is, $X_{1} \perp Y$, in which case, $\min = \max = V + 0$. Finally, it is possible that $X_{1} \perp X_{2}$ so the upper bound is still $\min(V + U, 1)$.

If you knew more about the relationship between $X_{1}$ and $X_{2}$, I think you could say more.

  • 1
    $\begingroup$ (+1) But, note that it is not (quite) true that if $X_1$ and $X_2$ are orthogonal, then their individual $R^2$ values will sum when including both in the model. We also need them to be orthogonal to the all-ones vector $\mathbf 1$. Note that you can use $\LaTeX$ on this site for marking up the math. :) $\endgroup$
    – cardinal
    Jul 15, 2012 at 22:00
  • $\begingroup$ That is true. Thanks very much for the comments, and for pointing out that $\LaTeX$ can be used. I thought it might but had tried mathjax style escapes ( and [ for inline/equations. Writing just like I would in TeX worked like a charm :) $\endgroup$
    – Joshua
    Jul 16, 2012 at 3:35
  • $\begingroup$ From @Magot's answer below, what happens if $r_{1,Y}$ is negative, and $r_{1,2}$ is non-zero? We can find some setting here when $R^2$ is greater than $U+V$ and close to $1$, but $U$ and $V$ are less than say $0.3$ each. $\endgroup$
    – newbie
    Oct 27, 2021 at 4:04

Let $r_{1,2}$ equal the correlation between $X_1$ and $X_2$, $r_{1,Y}$ equal the correlation between $X_1$ and $Y$, and $r_{2,Y}$ the correlation between $X_2$ and $Y$. Then $R^2$ for the full model divided by $V$ equals

$$\left(\frac{1}{(1 - r_{1,2}^2)}\right) \left(1 - \frac{2 \cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} + \frac{U}{V}\right).$$

So $R^2$ for the full model equals $V$ only if $r_{1,2} = 0$ and $r_{1,Y}^2 = U = 0$ or

$$r_{1,2}^2 = \frac{2\cdot r_{1,2} \cdot r_{1,Y}}{r_{2,Y}} - \frac{U}{V}.$$

If $r_{1,2} = 0$, $R^2$ for the full model equals $U + V$.

  • $\begingroup$ (+1) Cute. Welcome to the site. Please consider registering your account so you can participate more fully. I'll have to look at this expression a little more closely later on. :) $\endgroup$
    – cardinal
    Jul 18, 2012 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.