1
$\begingroup$

For sklearn's Random forest classification module, setting max_features to none takes into consideration all the n features for building each tree. In this case, how is it different from applying bagging to simple CART. Also isn't feature sampling the USP of random forest.

$\endgroup$
  • 1
    $\begingroup$ Actually it uses $\sqrt{N}$ ($N$ being number of all features) by default, see documentation $\endgroup$ – Jakub Bartczuk Mar 25 '18 at 16:43
  • $\begingroup$ sorry I meant "none" $\endgroup$ – rp7 Mar 26 '18 at 18:44
  • 1
    $\begingroup$ Small correction: $m$ of the $p$ features are considered before each node, not before each tree, in Random Forest. Selecting a subset of predictors before each tree is the "random subspace" method, not Random Forest as typically understood/implemented. $\endgroup$ – dlid Jul 21 '18 at 10:44
1
$\begingroup$

In bagging, the only parameter we tune is the number of trees. In Random Forest, we tune both the number of trees and the number of input variables $m \leq p$ considered for splitting each node. If $m = p$, we are bagging. Typically we choose $m \ll p$.

I assume by "USP" you mean "unique selling proposition?" If so, yes, the ability to consider a subset of features at each node is the main feature differentiating Random Forest from Bagging. The fact that one could set $m=p$ doesn't mean one should, and it doesn't invalidate the random feature subsetting as a "selling point" of Random Forest.

One typically uses $\sqrt{p}$ features, at least for classification problems.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.