By hastie et al decision trees have low bias and high variance why does boosting work even though bias not being a problem for trees?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Trees are very flexible models, depending on their configuration.
Trees with large maximum depth have low bias and high variance. They are strong learners, ideal candidates for bagging.
Trees with small maximum depth (sometimes a single decision rule, otherwise known as "decision stumps") have high bias, but low variance. These are weak learners, ideal candidates for boosting.
So it really depends on how you setup them. Hastie et al. probably recommend the second group for boosting.
-
$\begingroup$ good, i know you're right but i'd like know where (book, article,...) you get this. I'm making a monography about boosting models. $\endgroup$ Jul 22, 2021 at 22:47
-
-
$\begingroup$ @DaviAmérico I think in the very references you use to study about ensemble learning these facts are often stated, but I don't have a reference to offer you right now, might check it later (Yeah, I'm Brazilian :) ) $\endgroup$– FirebugJul 22, 2021 at 23:26