1
$\begingroup$

I am using sklearn to estimate a random forest classifier. Out of curiosity I have set max_features=None and max_depth=1. Everything else is left untouched.

I would expect the feature importance, which I get via feture_importances_ to consist of only 1 value. However, the feature_importance has values for all values of my features. How can that be possible and what am I missing?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

This is because a Random Forest consist not only in one decision tree but N, being N the parameter in n_estimators (https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html, https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)

Therefore, you will have n_estimators decision trees with max_depth=1. This allows you to have several feature importances.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Okay - but when I use boostrap=False, it should always use the same feature - right? $\endgroup$
    – freddy888
    Commented Mar 26, 2019 at 15:06
  • $\begingroup$ According to the documentation: "The sub-sample size is always the same as the original input sample size but the samples are drawn with replacement if bootstrap=True (default). If False, the whole datset is used to build each tree". But even if it uses the whole dataset, Random Forest will "search for the best feature among a random subset of features" (towardsdatascience.com/the-random-forest-algorithm-d457d499ffcd). This allows to different features to be selected for each Decision Tree $\endgroup$
    – LocoGris
    Commented Mar 26, 2019 at 15:11
  • 2
    $\begingroup$ What does max_features = none give in python? Does it give the default value? Because the default is to use square root of the number of features for each split, which would explain why OP is getting different splits. $\endgroup$
    – astel
    Commented Mar 26, 2019 at 15:17
  • 1
    $\begingroup$ I just tried it with boostrap=False and the feature importance is 1 for one feature and 0 for the others. That was the issue... $\endgroup$
    – freddy888
    Commented Mar 26, 2019 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.