I am running a binomial logistic regression with a dependent variable is 0/1 and the explanatory variables are categorical - binary and multiple categories. When I add an interaction term, and run a VIF test for presence of multicollinearity, several of the coefficients of predictor variables take a VIF value of 5 or greater. I have read that as a rule of thumb 5 indicates issues with multicollinearity and that the model should be respecified. Thoughts on whether that threshold is appropriate. Is there a rule of thumb recommended for a generalized variance inflation factor value to assess severe multicollinearity?
3
-
1$\begingroup$ The usual variance-inflation factor does not even describe the variance inflation in a logistic regression. Generalized variance-inflation factor is what you would want to calculate for your generalized linear model. // How are you using the logistic regression model? What do you want to learn by fitting it? If you just want to make predictions, variance inflation is not so important. $\endgroup$– DaveCommented Jun 2, 2023 at 1:17
-
$\begingroup$ The model is being used for conducting significance testing to test the association between outcome and explanatory variable of interest, and not for predictions, so represents a potential issue with precision of estimates. Is there a rule of thumb recommended for a generalized variance inflation factor value to assess severe multicollinearity? $\endgroup$– siliCommented Jun 2, 2023 at 1:30
-
2$\begingroup$ Interactions induce collinearity. You probably shouldn't worry about it. Or at least, center your predictors, create the interaction from the centered variables, then use all three of those variables in your model instead of the original ones. $\endgroup$– BigBendRegionCommented Jun 2, 2023 at 2:48
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Some of the trouble with going by the VIF is that there is more to the story than how the standard errors get inflated by multicollinearity, which is what VIF captures.
You could drop some of the variables that cause this variance inflation, but this puts you at a risk of increasing the residual variance. Since the coefficient standard error is a function of both the VIF and the residual variance, it is not a given that taking steps to lower the VIF actually works toward your goal of lowering the standard error; your steps to lower the VIF might be successful but lead to such an increase in the residual variance that the coefficient standard error winds up higher, despite the lower VIF.
Another issue to consider is omitted-variable bias. When you drop variables from a regression in order to lower multicollinearity, you risk biasing the estimates of the remaining coefficients. Thus, you can wind up with a low estimation variance but a high estimation bias. You might not want this. You might prefer more variance with less bias to little variance with considerable bias.
Consequently, there is much more to the story than just a VIF threshold of, say, $5$ or $10$ like sometimes get recommended in introductory courses.
Finally, the usual variance-inflation factor from OLS linear regression does not describe how the standard errors are inflated by multicollinearity in a logistic regression. Generalized variance-inflation factors would be considered for generalized linear models.
