0
$\begingroup$

I would like to ask a basic question. A multiple regression model, y ~ x1 + x2 + x3, has R2 value of 0.152. Now I have a new variable z, and its linear regression model, y ~ z, has R2 value of 0.051. However, when I put z in the multiple regression model, y ~ x1 + x2 + x3 + z, then R2 value is 0.153. Why is it not 0.152 + 0.051?

$\endgroup$
0
$\begingroup$

Remember that $R^2$ is the percent of variability in $y$ that is explained by your independent variables. Based on your data, this means that $z$ on its own explains about $5.1\%$ of the variability in $y$ but that it only explains $0.01\%$ of the variability in $y$ once $x_1$, $x_2$, and $x_3$ have already been taken into account.

Depending on the statistical software you use, this might be different. It will depend on which "sums of squares" each software uses. R, for example, uses sequential sums of squares, which means that the first independent variable $x_1$ in will explain some of that variability in $y$, then the next independent variable $x_2$ may explain some more variability in $y$ or it might not once $x_1$ is already taken into account. Thus the $R^2$ when adding in a new variable $z$ will not necessarily increase by the same $R^2$ value when you merely regress $y$ on $z$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Dear Matt, thank you very much for your answer. Does it make sense to think that (5.1 - 0.01)% of the variability is explained by the other predictor variable(s) when z is added into the model where x1, x2 and x3 are already present? $\endgroup$ – Jatin Dec 10 '15 at 20:09
  • $\begingroup$ That's correct. We don't know how much is explained by x1, x2, or x3 individually without more analysis, but they jointly explain about 5.09% of the variability in y. $\endgroup$ – Matt Brems Dec 10 '15 at 22:24
  • $\begingroup$ It's probably worth pointing out that this effect is due to the fact that $z$ is correlated with one or more of the other explanatory variables. In other words, once you're already taking the other variables into account, only the component of $z$ which is orthogonal to all of the other variables contributes to explaining variability in $y$. OP, please accept the answer provided by @MattBrems if you find it acceptable. $\endgroup$ – Josh Dec 10 '15 at 22:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.