I am running a Logistic regression on 6 independent variables and running chisquare tests shows high degree of association between 4 independent variables. Most of the topics suggest running first uni-variate regression and then keep on adding variables. I havent got any good guide for how to run this. PLease help!
To add to Maarten Buis's answer:
You may consider checking VIF (Variance Inflation Factor). Or it's generalized version GVIF if you have categorical independent variables (IVs).
You first put all your IVs into regression and remove the one with greatest VIF. Then redo calculations, and proceed until yoy have only IVs with (G)VIF below 10 or 5 (you can find different guidelines for cut-off value)
More detailed discussion about GVIF is here:
It really depends.
First, how high is high? Multicollinearity is not a problem, as long as the association is not perfect. It results in less statistical power, but that is an accurate representation of the amount of information present in the data.
Second, is the association high because they are different measurements of the same underlying concept, or are they different but related concepts? In the former case it would make sense to combine the items and use that to get a better estimate of the underlying concept. In the latter case you just add the variables all at once if you think you need to add, and you will just have to live with the fact that that will cost you statistical power.