3
$\begingroup$

Can I ignore multicolinearity problem if all the regression coefficients are highly significant?

My data is large enough (i.e. I have several regression models where each of the data points for them ranges from 2958 to 11646 data points for every each 6 independent variables. so it is 6 times of these 2958 - 11646 data points for each independent variable to count the total number of data points) and all the resulting coefficients are significant enough in less than 0.01 level. The only thing I see is that one of the variable has the correlation of 0.9 (i.e. the correlation value of one variable to another one is 0.9 but I do not want to remove either of them.).

I am trying to see on unit increase effect of this variable while keeping all other variables constant. Can I keep this variable?

Besides, if I delete one of the variable with high VIF which is between 13 anad 14, all the other VIF are safe but the intercept becomes insignificant for all cases

I am also referring to the following website comment: http://www.researchconsultation.com/multicollinearity-multiple-regression.asp

So, in sum, my ultimate goal is to use the final output from the logistic regression model generated from the independent variables and one binary dependent variable. If so, do you think I can ignore the multicolinearity problem?

Improve this question
$\endgroup$
3
  • $\begingroup$ "Has the correlation of 0.9"? Can you please elaborate on the meaning of this phrase? How did you conclude this? How big is your sample and what is the number of parameters in your model? $\endgroup$
    – usεr11852
    Commented Jun 22, 2016 at 18:00
  • $\begingroup$ Of course. Please check now. $\endgroup$
    – Eric
    Commented Jun 22, 2016 at 18:05
  • $\begingroup$ The numerical output is the one I need and want to show how this final numerical output from my different model other than my logistic regression model which is a function of y gets different with different emphasis on indepdent variable increase up to two units. $\endgroup$
    – Eric
    Commented Jun 24, 2016 at 17:31

3 Answers 3

Reset to default
10
$\begingroup$

What is your ultimate goal? Are you going to use the model to make predictions on the mean response? If so, correlation is not a problem. However, if you want to make inference then you have to think about it.

I woulg suggest to look up VIF (variable inflation factor) and see whether there is really multicollinearity.

p.s. I could not comment since I've just signed up and have to reach minimum number of reputations.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – whuber ♦
    Commented Jun 23, 2016 at 16:45
  • 1
    $\begingroup$ I would add a point. Multicolinearity is not a problem in a prediction context if only the correlation between your predictors doesn't change! $\endgroup$
    – Metariat
    Commented Jun 24, 2016 at 19:32
7
$\begingroup$

You can ignore multicollinearity for a host of reasons, but not because the coefficients are significant. In fact, it's one of the issues and manifestations of the multicollinearity issue when you have two or more variables which highly significant when put into the regression together, and not significant at all when added one by one.

For instance, in econometrics you'll get very significant coefficients if you have cross exchange rates, e.g. British pound to dollar, Euro to Dollar and Swiss mark to Dollar, while individually they may not be significant in your model.

Improve this answer
$\endgroup$
12
  • $\begingroup$ Oh sorry. But these are the VIF values that I get from the linear regression model while I am trying to use logistic regression model. Is it safe? $\endgroup$
    – Eric
    Commented Jun 22, 2016 at 19:10
  • $\begingroup$ the VIF values of the logistic regression is above 10 that I got by imposing VIF function right over the logistic regression in R-programming. However, the VIF values of the linear regression model is reasonable as I've described above. If I am using logistic regression model, should I worry about this? $\endgroup$
    – Eric
    Commented Jun 22, 2016 at 19:15
  • 2
    $\begingroup$ You're looking for a mechanical decision rule. There are no absolute rules here. You can go with one of the rules of thumb like 10 or 5, but they're all shaky grounds. You have to make a judgement call based on entirety of evidence, where simple rules can be an input into your decision. $\endgroup$
    – Aksakal
    Commented Jun 22, 2016 at 19:48
  • $\begingroup$ Besides, my logistic regression has all significant coefficients p-values all lower than 0,01 when put into one logistic regression together at the same time. Not separate as you mentioned above. If so, do I still need to worry about multicolinearity if VIF of these two are 13 ~ 14 or so? $\endgroup$
    – Eric
    Commented Jun 23, 2016 at 0:48
  • $\begingroup$ VIF basically does what I described: calculates the variance inflation in absence of correlation. You really need to understand the tests that you're planning to use. Again, whether you have to worry or not cannot be based on just one test. VIF 13 would be considered too high by many. $\endgroup$
    – Aksakal
    Commented Jun 23, 2016 at 1:09
2
$\begingroup$

Luckily, there is a diagnostic tool called the variance inflation factor (VIF) that allows you to assess whether you have a multicollinearity problem. Usually, VIF scores > 10 are a cause for concern.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – whuber ♦
    Commented Jun 23, 2016 at 14:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.