3
$\begingroup$

An interesting question my friend told me: I have two simple linear regression models with R$^2$ = 0.1

Y ~ X1
Y ~ X2

If I combine these two models, Y ~ X1 + X2, is it possible that R$^2$ > 0.2? If so, can you provide an example?

$\endgroup$
1
  • $\begingroup$ Why is that a surprise? $\endgroup$
    – SmallChess
    Oct 17 '17 at 5:46
7
$\begingroup$

Yes.

This doesn't get your numbers on the nose, but the spirit is the same

df <- data.frame(
  x1 = c(0, 0, 0, 1, 1, 1),
  x2 = c(0, 1, 2, -1, 0, 1),
  y = c(0, 2, 4, 2, 4, 6)
)

With model $R^2$ statistics

> summary(lm(y ~ x1, data=df))$r.squared
[1] 0.2727273
> summary(lm(y ~ x2, data=df))$r.squared
[1] 0.2066116
> summary(lm(y ~ x1 + x2, data=df))$r.squared
[1] 1
$\endgroup$
2
  • $\begingroup$ This example shows that the R$^2$ for the sum of the Xs can be greater than the sum of the two R$^2$s. As you say you can probably manipulate the data values to make the original two R$^2$s =0.1 and make the R$2$ for the sum greater than 0,2. But since the sum in your example is (0,1,2,0,1,2) and y=(0,2,4,2,4,0) how can the R$^2$=1? There is not perfect correlation. Is x1+x2 in the code actually the sum or is it the least squares linear combination of x1 and x2 including an intercept? $\endgroup$ Oct 17 '17 at 5:50
  • 2
    $\begingroup$ y ~ x1 + x2y is not a sum but R code for: explain y linearly through x1, x2 and an intercept. This is a OLS model with an intercept, even though this intercept happens to be 0. It's $y\leftarrow 4\times x_1+2\times x_2 + 0 + \epsilon$. with $\epsilon=0$. $\endgroup$
    – Bernhard
    Oct 17 '17 at 6:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.