By hastie et al decision trees have low bias and high variance why does boosting work even though bias not being a problem for trees?


1 Answer 1


Trees are very flexible models, depending on their configuration.

  • Trees with large maximum depth have low bias and high variance. They are strong learners, ideal candidates for bagging.

  • Trees with small maximum depth (sometimes a single decision rule, otherwise known as "decision stumps") have high bias, but low variance. These are weak learners, ideal candidates for boosting.

So it really depends on how you setup them. Hastie et al. probably recommend the second group for boosting.

  • $\begingroup$ good, i know you're right but i'd like know where (book, article,...) you get this. I'm making a monography about boosting models. $\endgroup$ Commented Jul 22, 2021 at 22:47
  • $\begingroup$ uauhaUHSAUH vc é br $\endgroup$ Commented Jul 22, 2021 at 23:11
  • $\begingroup$ @DaviAmérico I think in the very references you use to study about ensemble learning these facts are often stated, but I don't have a reference to offer you right now, might check it later (Yeah, I'm Brazilian :) ) $\endgroup$
    – Firebug
    Commented Jul 22, 2021 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.