" The fundamental difference between bagging and random forest is that in Random forests, only a subset of features are selected at random out of the total and the best split feature from the subset is used to split each node in a tree, unlike in bagging where all features are considered for splitting a node." Does that means that bagging is the same as random forest, if only one explanatory variable (predictor) is used as input?
The fundamental difference is that in Random forests, only a subset of features are selected at random out of the total and the best split feature from the subset is used to split each node in a tree, unlike in bagging where all features are considered for splitting a node.
Bagging in general is an acronym like work that is a portmanteau of Bootstrap and aggregation. In general if you take a bunch of bootstrapped samples of your original dataset, fit models $M_1, M_2, \dots, M_b$ and then average all $b$ model predictions this is bootstrap aggregation i.e. Bagging. This is done as a step within the Random forest model algorithm. Random forest creates bootstrap samples and across observations and for each fitted decision tree a random subsample of the covariates/features/columns are used in the fitting process. The selection of each covariate is done with uniform probability in the original bootstrap paper. So if you had 100 covariates you would select a subset of these features each have selection probability 0.01. If you only had 1 covariate/feature you would select that feature with probability 1. How many of the covariates/features you sample out of all covariates in the data set is a tuning parameter of the algorithm. Thus this algorithm will not generally perform well in high-dimensional data.
I would like to provide clarification, there is a disctinction between bagging and bagged trees.
Bagging (bootstrap + aggregating) is using an ensemble of models where:
- each model uses a bootstrapped data set (bootstrap part of bagging)
- models' predictions are aggregated (aggregation part of bagging)
This means that in bagging, you can use any model of your choice, not only trees.
Further, bagged trees are bagged ensembles where each model is a tree.
So, in a sense, each bagged tree is a bagged ensemble, but not each bagged ensemble is a bagged tree.
Given this clarification, I think that user3303020's answer provides a good explanation.