Is multicolinearity problem ignorable under this situation?

Question

When I run the logistic regression, two independent variables have VIF values greater than 10 like 13 or so. Logistic regression is the one I will use to measure the overall change in the dependent variable with one incremental changes in every each independent variable. However, when I run the linear regression model version on my same data, the VIF values I get from all the independent variables are either slightly greater than 1 or around 5 or 6. Besides, all the coefficients are highly significant all below 0.01.

If so, is it safe to use my logistic regression model to find out the incremental effect of every each independent variable without dropping out any independent variables?

If my purpose is to find an incremental effect of every each independent variable on the overall dependent value in a logistic regression model, is multicolinearity an important problem to consider? Can I ignore it?

If my p-values are all less than 0.01 which is the same indicator as t-value, will I still need to worry about colinearity issue even though my VIF scores for two variables are around 13-14? Based on what you said, if p-value is safe enough, will this be still a problem?

I am also referring to the following website comment: http://www.researchconsultation.com/multicollinearity-multiple-regression.asp

So, in sum, my ultimate goal is to use the final output from the logistic regression model generated from the independent variables and one binary dependent variable. If so, do you think I can ignore the multicolinearity problem?

Answer 1

Collinearity is a problem only if the model needs to enforce an assumption of the independence of predictors. It's known that the model coefficients usually aren't affected by collinearity, whereas std errors and t-values are. This can and will bias the model selection process. Given your objectives, this may or may not be important. Examples of where IV independence can be important include pricing models where the analyst needs to know what the impact of price changes are clear and free of the influence of the other predictors in the model.

One of the biggest problem with the VIF is that there are no reliable guidelines as to when there is a real problem. In other words, beginning with Regression Diagnostics the 80s book by Belsey, Kuh and Wallace, different authors in different pubs propose differing cutoffs in the VIF to indicate collinearity.

There are more checks for collinearity than the VIF. For instance, are the signs of the parameters consistent with a pairwise correlation? If the sign is reversed, that suggests that the predictor is strongly related with at least one other IV in the model. Have you looked at the partial and semi-partial correlations among your model features? They, too, are useful diagnostic metrics insofar as they can identify where a problem might be. Another thing to look for is endogeneity, one check for which is non-zero association between the residual and each IV.

While all of these indicators and metrics diagnose and suggest a potential problem, none of them are confirmatory. To the best of my knowledge, no such confirmatory test for collinearity exists.