# How multicolinearity affect the prediction

In a linear regression model, if some of the predictors are correlated, then in the output of most software, you will see very large p-value in those coefficients and very high standard error. My question is: does this matter in terms of the prediction?

I made a small example myself. I create the following dataset

$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \epsilon$, where $\epsilon\sim~N(0,1)$ which is independent of $X_1,X_2$ and $X_2 = aX_1+b$, i.e., $X_1$ and $X_2$ are perfectly correlated. So after rearrangement, we see that $Y=\beta_0+b\beta_2 + X_1(\beta_1+a\beta_2)+\epsilon$. Then I use R to run linear regression. Not surprise, R reports NA for the coefficient, standard error and p value of $X_2$. But then I look at the estimated coefficient of $X_1$, it is very close to $\beta_1+a\beta_2$ and the estimated intercept is also very close to $\beta_0+b\beta_2$.

We all know that the estimator of the coefficient $\theta=[\beta_0,\beta_1,\beta_2]$ has the form $\hat{\theta}=(X^{T}X)^{-1}X^TY$. In this case the matrix $X^{T}X$ is not invertible. I wonder how R reports correct estimate for $\beta_1+a\beta_2$ and $\beta_0+b\beta_2$ in this case? Moreover, if I only care about prediction and don't care about interpretation of the model, can I just ignore those predictors that has NA or big standard error of the coefficients estimator and just use the remaining model?

A more general question, if I made my data such that $x_2$ and $x_1$ has some correlation but not perfect, then will the LS method be able to derive a consistent estimator of the coefficients? If it can't then will there be a method achieve this goal?

• I think you may benefit by splitting up your questions into more than one post. There are five questions which are not necessarily dependent on each other. You may get better answers when you stay focused. – Richard Hardy Feb 17 '15 at 18:38
• Slightly off-topic, but Dave Giles has a nice (but longish) blog post on multicollinearity. – Richard Hardy Feb 17 '15 at 18:59

Your question in the first paragaph has been asked and answered before at Cross Validated. You will find answers by searching for prediction and multicollinearity. See e.g. here. If the answers are not sufficient, try to explain what exactly you are missing.