3
$\begingroup$

" The fundamental difference between bagging and random forest is that in Random forests, only a subset of features are selected at random out of the total and the best split feature from the subset is used to split each node in a tree, unlike in bagging where all features are considered for splitting a node." Does that means that bagging is the same as random forest, if only one explanatory variable (predictor) is used as input?

$\endgroup$
3
$\begingroup$

Bagging in general is an acronym like work that is a portmanteau of Bootstrap and aggregation. In general if you take a bunch of bootstrapped samples of your original dataset, fit models $M_1, M_2, \dots, M_b$ and then average all $b$ model predictions this is bootstrap aggregation i.e. Bagging. This is done as a step within the Random forest model algorithm. Random forest creates bootstrap samples and across observations and for each fitted decision tree a random subsample of the covariates/features/columns are used in the fitting process. The selection of each covariate is done with uniform probability in the original bootstrap paper. So if you had 100 covariates you would select a subset of these features each have selection probability 0.01. If you only had 1 covariate/feature you would select that feature with probability 1. How many of the covariates/features you sample out of all covariates in the data set is a tuning parameter of the algorithm. Thus this algorithm will not generally perform well in high-dimensional data.

$\endgroup$
2
$\begingroup$

The fundamental difference is that in Random forests, only a subset of features are selected at random out of the total and the best split feature from the subset is used to split each node in a tree, unlike in bagging where all features are considered for splitting a node.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.