# Is multicollinearity the issue here?

How is that I get a low $R^2$ if I use $X1$ against $Y$ and $X2$ against $Y$, but a $R^2 =1$ if I use both together?

How does the combination blow out the $R^2$?

Could you please show me that calculation?

The data:

Y   12.37   12.66   12.00   11.93   11.06   13.03   13.13   11.44   12.86      10.84    11.20   11.56   10.83   12.63   12.46

X1  2.23    2.57    3.87    3.10    3.39    2.83    3.02    2.14    3.04    3.26    3.39    2.35    2.76    3.90    3.16

X2  9.66    8.94    4.40    6.64    4.91    8.52    8.04    9.05    7.71    5.11    5.05    8.51    6.59    4.90    6.96


In multiple regression, you are trying to model observations, $y$ that are supposed to be noisy versions of a prediction, $X\beta$. That is, we should have: $$y=X\beta+\epsilon$$

In your case, $\epsilon$ is almost zero. That is, both explanatory variables almost perfectly explain your observations. You have: $$\beta = (-4.515413640943276 \text{ (constant offset)}, \\ 3.097007886142981, \\ 1.031859030944453)$$ This gives you a nearly perfect fit. Regarding multicolinearity: By definition, multicolinearity occurs when your predictor variables, $X1$ and $X2$, are linearly related.

Now, in your case, $X1$ and $X2$ do, in fact, have a high correlation coefficient of about -$0.9$ (I had missed this in a previous edit). However, this is not the cause of the $R2$ jump that you observe by including both $X1$ and $X2$.

For, let multicolinearity have been present. Then, we could write for a small error $\eta$: $$X2 = a + bX1 + \eta$$ In that case, we would be able to write, for each data point $i$: $$y^{(i)} = \beta_0 + \beta_1 X1^{(i)} + \beta_2 (a + bX1^{(i)} + \eta^{(i)}) + \epsilon^{(i)}$$ $$\Rightarrow y^{(i)} = (\beta_0 + a \beta_2) + X1^{(i)} (\beta_1 + b \beta_2) + \epsilon^{(i)} + \beta_2 \eta^{(i)}$$

Now that we know that $\epsilon^{(i)}$ are nearly zero, then, unless $\beta_2$ were large, had multicolinearity been an issue we would have seen $R2 = 1$ for even the $Y-X1$ regression. Instead, we get $R2 \approx 0$. This means that while $X1$ and $X2$ together are good predictors, they aren't linearly related enough to be good predictors individually.

Having said that, multicolinearity might play a role in other aspects of your problem, given the $X1-X2$ correlation coefficient of $-0.9$.

• Thanks but as both variables are highly correlated, why do you say there's no collinearity? Why could each variable have so small predictive power alone but perfectly fit together? – Luis Sep 14 '16 at 12:42
• I am having trouble reconciling the 3D plot with the scatterplots. The former appears to show linear relations among all three variables whereas the latter show something entirely different. – whuber Sep 14 '16 at 12:59
• @whuber, the plots are right. – Luis Sep 14 '16 at 13:35
• "both variables are highly correlated": which two do you have in mind? $X1$ and $X2$ are not correlated. The reason I said there is no multi-colinearity is because of the definition of the term: "Multicollinearity exists when two or more of the predictors in a regression model are moderately or highly correlated." The 3 variables are related by a linear equation but it doesn't qualify as multicolinearity as per the definition. – Salmonstrikes Sep 14 '16 at 21:58
• @whuber: the 2D scatters are just projections of the 3D one on each plane of the planes. What I was trying to show in the 3D plot was that all the $(X1, X2, Y)$ scatter points lie in a plane (linearly related), yet pairwise, they are not correlated (esp. the $X2-X1$ plot) – Salmonstrikes Sep 14 '16 at 21:59