3
$\begingroup$

When I run the logistic regression, two independent variables have VIF values greater than 10 like 13 or so. Logistic regression is the one I will use to measure the overall change in the dependent variable with one incremental changes in every each independent variable. However, when I run the linear regression model version on my same data, the VIF values I get from all the independent variables are either slightly greater than 1 or around 5 or 6. Besides, all the coefficients are highly significant all below 0.01.

If so, is it safe to use my logistic regression model to find out the incremental effect of every each independent variable without dropping out any independent variables?

If my purpose is to find an incremental effect of every each independent variable on the overall dependent value in a logistic regression model, is multicolinearity an important problem to consider? Can I ignore it?

If my p-values are all less than 0.01 which is the same indicator as t-value, will I still need to worry about colinearity issue even though my VIF scores for two variables are around 13-14? Based on what you said, if p-value is safe enough, will this be still a problem?

I am also referring to the following website comment: http://www.researchconsultation.com/multicollinearity-multiple-regression.asp

So, in sum, my ultimate goal is to use the final output from the logistic regression model generated from the independent variables and one binary dependent variable. If so, do you think I can ignore the multicolinearity problem?

Improve this question
$\endgroup$
3
$\begingroup$

Collinearity is a problem only if the model needs to enforce an assumption of the independence of predictors. It's known that the model coefficients usually aren't affected by collinearity, whereas std errors and t-values are. This can and will bias the model selection process. Given your objectives, this may or may not be important. Examples of where IV independence can be important include pricing models where the analyst needs to know what the impact of price changes are clear and free of the influence of the other predictors in the model.

One of the biggest problem with the VIF is that there are no reliable guidelines as to when there is a real problem. In other words, beginning with Regression Diagnostics the 80s book by Belsey, Kuh and Wallace, different authors in different pubs propose differing cutoffs in the VIF to indicate collinearity.

There are more checks for collinearity than the VIF. For instance, are the signs of the parameters consistent with a pairwise correlation? If the sign is reversed, that suggests that the predictor is strongly related with at least one other IV in the model. Have you looked at the partial and semi-partial correlations among your model features? They, too, are useful diagnostic metrics insofar as they can identify where a problem might be. Another thing to look for is endogeneity, one check for which is non-zero association between the residual and each IV.

While all of these indicators and metrics diagnose and suggest a potential problem, none of them are confirmatory. To the best of my knowledge, no such confirmatory test for collinearity exists.

Improve this answer
$\endgroup$
12
  • $\begingroup$ Thank you so much. If my p-values are all less than 0.01 which is the same indicator as t-value, will I still need to worry about colinearity issue even though my VIF scores for two variables are around 13-14? Based on what you said, if p-value is safe enough, will this be still a problem? $\endgroup$
    – Eric
    Jun 23 '16 at 8:25
  • $\begingroup$ I am also referring to the following website comment: researchconsultation.com/… My main purpose is to use the dependent variable as one of my input. But I do this while keeping all other independent variables increase by one unit only the variable of my interest increases by 2 unit and I use the resulting dependent variable value for my input. So I have a mixed feeling about the comment on the website link above $\endgroup$
    – Eric
    Jun 23 '16 at 9:49
  • 1
    $\begingroup$ Where in my response did I say that "p-value is safe enough?" I did say that there is no cookbook recipe for deciding when there is a problem with collinearity $\endgroup$
    – Mike Hunter
    Jun 23 '16 at 10:06
  • $\begingroup$ Thanks but my question is if my purpose is to use the dependent variable as my input where I assign value 1 for all the independent variable except one independent variable which I assign value 2 to measure its impact where I only use the overall dependent variable for my input and if I do this for every each indepdendent variable, do I still need to worry about multicolinearity issue? $\endgroup$
    – Eric
    Jun 23 '16 at 10:38
  • $\begingroup$ Sorry. I don't understand the question $\endgroup$
    – Mike Hunter
    Jun 23 '16 at 10:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.