# how does multicollinearity affect feature importances in random forest classifier?

I have a random forest binary classifier, but the results from the feature importances are somewhat erratic. Here's what I want to know: Does multicollinearity mess up feature_importances_ in a RandomForestClassifier?

I'm using sci-kit learn (sklearn in python) for the random forest classifier, and getting the feature importances.

To validate the output of feature_importances_ from the RandomForestClassifier in sklearn, I tried removing the most important feature each time (i.e. the feature with the highest feature importance, to see if the second most important feature would appear as the most important feature in the next iteration.... But, this never happened.

The results of this were very erratic, and the order of feature importances was not preserved. At every iteration, there was one feature with extremely high importance (like 0.7 or 0.8), all of the others between 0.1 and 0.0001. There were 9 features to start. The second highest feature importance never appeared as most important in the next iteration.

Does multicollinearity mess up the feature importances, or is there something else I'm missing that messes this up?

• How do I know which of the variables is part of the multicollinearity? Is PCA a way to do this? Also, I guess it isn't clear to me how to determine which one to remove after the multicollinearity has been identified. Jan 5 '16 at 5:44