4
$\begingroup$

I'm plotting my RF feature importance using the method shown here.

Where the error bars in the plot are defined by:

std = np.std([tree.feature_importances_ for tree in forest.estimators_],
             axis=0)

However, the resulting error bars are huge, and even range into the negative range. I'm not quite sure what my interpretation of this should be?RF feature importances + error bars. Does this mean that the inter-tree variation of this variable's importance is huge, and if so does that mean that the variable is in fact not a great predictor of the target variable?

$\endgroup$
1
$\begingroup$

It would be helpful to know more about your problem and your implementation. Especially, if you are using the defaults of the RF classifier you only have 10 trees in your ensemble. If you increase the number of estimators I would expect the std to get smaller.

If you have highly correlated features, then each variable is equally suited to be used in a split. Therefore, there is large variation in the importance of a variable in each tree. Lastly, before worrying about the negative values you can also look at all the actual values to better estimate how they actually distribute.

If you did everything correctly, it indeed indicates that there is not a single good variable for predicting your classes but rather a collection of features that is needed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.