1
$\begingroup$

I am estimating a linear regression: $Y=f(X_1,X_2,X_3,X_4,X_5)$. My test shows that when the equation includes $X_4$ and $X_5$ only, $X_4$ is not statistically significant ($t$-value=1.26). However, When $X_1$,$X_2$,and $X_3$ are added, its significance is increased to significant level ($t$-value =2.36). In another words, the significance of $X_4$ depends on the presence of $X_1$, $X_2$ and $X_3$ in the model.

I tested the correlation between $X_4$ and $X_1$, $X_2$, and $X_3$. Bivariate correlations between them are all below 0.5. If I understand it correctly, multicollinearity should not be a concern at such correlation level.

My question is: Should I drop X4 from my final model? If no, how should I explain its significance?

$\endgroup$
  • 1
    $\begingroup$ Multicollinearity is not necessarily a function of a single bivariate correlation. In addition to @Greg's answer, you might look at other posts here tagged multicollinearity, including this one. $\endgroup$ – ttnphns Sep 26 '13 at 19:38
  • $\begingroup$ Can you clarify what you mean by "unstable coefficient" in the title? The fact that the significance level (ie p-value) changes need not have anything to do with multicollinearity, & I wouldn't use the term "unstable" myself. Is the coefficient estimate similar (just more significant), or is it wildly different? $\endgroup$ – gung - Reinstate Monica Sep 26 '13 at 20:11
4
$\begingroup$

There are a few different things that could be going on here:

One, the multicolinearity cannot be assessed based only on bivariate correlations. Consider the case where x1, x2, and x3 are all generated as independent normal random vanriables and x4 is the sum of x1, x2, and x3. There is definite colinearity with x4, but the bivariate colinearities are not strong (I did a quick example and the correlations ranged from 0.55 to 0.63). So you should use a better measure of colinearity than just bivariate correlations.

Two, it may be that x1, x2, and x3 explain quite a bit of the variation in the response variable and are independent of x4, so that without x1-x3 the effect size of x4 is small compared to the residual variation, but when you include x1-x3 the residual variation is much smaller and the effect size of x4 stays about the same, but is now large relative to the residual variation and therefore significant.

There are probably other explanations as well. You should really explore your data and all the relationships, then also consider the science behind the data, which model to use should be the one that makes the most sense scientifically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.