Random forests are built by using a bagging algorithm and adding to it a variable selection feature that ensures that the trees built are really independent. More explicitly, during the construction of every tree, at each node it selects randomly among the p predictors a subset of m of them and uses only theses ones to decide of the split it is going to make. The trees therefore constructed are more uncorrelated in the cases where a few predictors of the data set are very strong with respect to the others in predicting the response.

The idea is that with respect to bagged trees it has lower variance, but if there is indeed a real dominance of some predictors with respect to the other in the prediction of the response, then it seems to me that ignoring this fact would lead to higher bias and therefore probably to higher error rate than bagged trees method, am I right ?

  • $\begingroup$ the parameter for deciding the number of trees in random forest should take care of this. $\endgroup$ – show_stopper Nov 29 '17 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.