Could multicollinearity be messing up my logistic regression? Can I overcome it?

Question

My data has 5 binary dependent variables, 9 categorical independent variables, and 3 continuous independent variables, with a sample size of 1232. The 5 dependent variables are just different ways of measuring enrollment persistence, specifically Next Subsession Persistence, Next Term Persistence, Next Year Persistence, Next Year Same Subsession Persistence, and Anytime Persistence. Two of the continuous independent variables may be correlated, since they are logically related - Incoming GPA and First Term GPA. Incoming GPA is high school GPA for the 100 first-time freshmen entering the program but transfer GPA for the 1132 transfer students entering the program.

I checked correlations of the continuous independent variables in SPSS and found that the Incoming GPA and First Term GPA correlation is only 0.115 with significance < 0.001, which I suspect is due to the large n or N, but it makes sense that they should be correlated.

Here's the question. Could multicollinearity be messing up the Logistic regression analysis here? Can I overcome it?

It's just that I'm finding that Logistic regression done separately for each of the 5 different independent variables is finding 2 of the categorical independent variables, which are Change of Major and Fulltime Enrollment, highly significant but in differing ways for the different types of persistence. Sometimes Change of Major is highly significant, sometimes Fulltime Enrollment is highly significant, sometimes both are. Also, neither Incoming GPA nor First Term GPA is significant in any of the models but I'm concerned about what may be a high correlation between them that I can't identify. I'm thinking that they must be correlated but their correlation is low and its high significance might be due to the high n.

Could multicollinearity be messing things up here? After reading several things off and on this site, I'm considering running a linear regression in SPSS to get multicollinearity VIFs even though what I'm going to ultimately perform will be logistic regression models. If I do that to reduce the number of independent variables in the final logistic regression analysis, I'm thinking it might be easier to interpret and the results might be more consistent. I can't find literature saying this is ever done but it seems like a good idea to me.

Answer 1

Multicollinearity might mess things up and might not. It really depends on what you're trying to do and what do you consider as risks...

If you want to use Logistic Regression for prediction (classification): Multicollinearity won't distort the model's output. You'll have the same numbers with features' removal or without it. One thing that's many times overlooked, is that multicollinearity might lead to numerical errors, as there's no limit to how big your coefficients may be. If one of your coefficients is at 1,000,000 the tiniest gap could distort your model's performance.

If you're using Logistic Regression as a descriptive or inference model: Multicollinearity is a serious issue. Your coefficients would probably not make any sense.

Overall, many scholars advice to remove features having perfect multicollinearity, as it may only harm your model. Reduction of features with a high VIF is one common way, there are plenty others.