I have a random forest binary classifier, but the results from the
feature importances are somewhat erratic. Here's what I want to know: Does multicollinearity mess up
feature_importances_ in a RandomForestClassifier?
sci-kit learn (
sklearn in python) for the random forest classifier, and getting the feature importances.
To validate the output of
feature_importances_ from the RandomForestClassifier in sklearn, I tried removing the most important feature each time (i.e. the feature with the highest feature importance, to see if the second most important feature would appear as the most important feature in the next iteration.... But, this never happened.
The results of this were very erratic, and the order of feature importances was not preserved. At every iteration, there was one feature with extremely high importance (like 0.7 or 0.8), all of the others between 0.1 and 0.0001. There were 9 features to start. The second highest feature importance never appeared as most important in the next iteration.
multicollinearity mess up the feature importances, or is there something else I'm missing that messes this up?