I have a multiple logistic regression with 11 independent variables (x1 to x11). I have another 4 continuous IVs that are highly correlated with each other and correlated with x11. The 4 IVs along with x11 measure different KPIs but they are essentially related in some sense hence the correlation. I cannot include them all because of multicollinearity issues.
X11 was included in the model since it gave the highest predictive performance on multiple test samples (in my case here, lift in the top decile).
I am being asked to measure the relative importance of each of the 4 IVs and x11 (rank them maybe) in relation to the dependent variable.
I have trying recycling them in the model and capture their coefficients but I know that it’s not an accurate comparison since the coefficients of the other 10 variables slightly change.
I have thought about contribution to the log-likelihood of the model which will help me rank them but won’t tell me about enough about they relate to the dependent variable.
Your guidance would be much appreciated.