I thought it would be interesting to talk about two of the best ensemble methods off-the-shelf: Random Forests and Boosting.
- When would you apply one method rather than the other one?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Just to start, a quick thought.
Random Forest can run in parallel and they are much faster to train, Boosting is an iterative algorithm instead. However, Boosting might converge early iteration-wise.
Boosting might overfit when there are many noisy features but Random Forest also does.
On the other hand, their target is almost similar: produce many different weak learners as much different as possible from each others. Random Forests tackle the problem with randomization and Boosting focuses on mis-classified examples of previous models to build a different one.