Subset Differences between Bagging, Random Forest, Boosting?

Question

Per my understanding, there are 2 kinds of "subsets" that can be used when creating trees: 1) Subset of the dataset, 2) Subset of the features used per split.

The concepts that I'm comparing are: 1) Bagging, 2) Random Forest, and 3) Boosting.

Please let me know if the following is correct or incorrect:

Bagging: Uses Subset of the dataset (bootstrapping) to create trees, but All features can be used for splits.

Random Forest: Uses Subset of the dataset (bootstrapping) to create trees, and only Subsets of features can be used for splits.

Boosting: Uses Entire dataset to create trees (no bootstrapping), and All features can be used for splits.

Yes I know there are more differences between the three, but wanted to hone in on these subset differences here and clarify my understanding on these first.

Answer 1

Random forests use bagging (bagging is just a contraction of "bootstrap aggregation," see Elements of Statistical Learning Section 8.7). Bagging draws a bootstrap sample of the data (randomly select a new sample with replacement from the existing data), and the results of these random samples are aggregated (because the trees' predictions are averaged). But bagging, and column subsampling can be applied more broadly than just random forest. (There's also a discussing in ESL of how random forest is well-positioned to benefit from bagging, while other learning methods are not.)

The boosting implementations that I'm familiar (e.g. xgboost) will also support random subsampling of columns. But your guess is correct, in the sense that column subsampling is really incidental to the boosting procedure itself. Boosting is about estimator $T+1$ "correcting" the errors of the previous $T$ estimators. Likewise, bagging is also incidental to the boosting procedure, but bootstrap resampling of the data could be implemented as well.