I have a multiple logistic regression with 11 independent variables (x1 to x11). I have another 4 continuous IVs that are highly correlated with each other and correlated with x11. The 4 IVs along with x11 measure different KPIs but they are essentially related in some sense hence the correlation. I cannot include them all because of multicollinearity issues.

X11 was included in the model since it gave the highest predictive performance on multiple test samples (in my case here, lift in the top decile).

I am being asked to measure the relative importance of each of the 4 IVs and x11 (rank them maybe) in relation to the dependent variable.

I have trying recycling them in the model and capture their coefficients but I know that it’s not an accurate comparison since the coefficients of the other 10 variables slightly change.

I have thought about contribution to the log-likelihood of the model which will help me rank them but won’t tell me about enough about they relate to the dependent variable.

Your guidance would be much appreciated.

  • $\begingroup$ How about partial correlation? $\endgroup$ – user2974951 Oct 16 '18 at 5:39
  • $\begingroup$ Partial correlation will not quantify the change in the dependent variable by changing the value of the covariate. It will certainly rank them though. $\endgroup$ – Moe Sam Oct 16 '18 at 15:31

Another possibility is to include all variables in the logistic regression model, and account for the multi-collinearity issues using ridge regression. This will appropriately penalize the coefficients of the correlates predictors and give you more stable results. I don't know which software you're using, but in R this is, for instance, available in the glmnet and arm packages.

  • $\begingroup$ Would the coefficients of the model have the same interpretation with ridge regression? I understand it penalizes the coefficients by adding some bias. Also, I am using R $\endgroup$ – Moe Sam Oct 16 '18 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.