Bagging classifier vs RandomForestClassifier Is there a difference between using a bagging classifier with base_estimaton=DecisionTreeClassifier and using just the RandomForestClassifier? This question refers to models from python library called sklearn.
 A: Yes, there is a difference. In sklearn if you bag decision trees, you still end up using all features with each decision tree. In random forests however, you use a subset of features.
The official sklearn documentation on ensembling methods could have been a bit more clear about the difference, here is what it says:


*

*"When samples are drawn with replacement, then the method is known as Bagging"

*"In random forests (see RandomForestClassifier and RandomForestRegressor classes), each tree in the ensemble is built from a sample drawn with replacement (i.e., a bootstrap sample) from the training set."


So it would appear there is no difference if you bag decision trees, right? It turns out, the documentation also states:
"Furthermore, when splitting each node during the construction of a tree, the best split is found either from all input features or a random subset of size max_features."
So this is one more way of introducing randomness, by limiting the number of features at the splits. In practice, it is useful to indeed tune max_features to get a good fit.
