I read somewhere that Gradiant boosting is a generalization of Adaboost. However, I cannot see why. Can Anyone elaborate?

  • 6
    $\begingroup$ AdaBoost can be formulated as gradient boosting with exponential loss. Have a look at chapter 10 of The Elements of Statistical Learning: web.stanford.edu/~hastie/ElemStatLearn $\endgroup$
    – sebp
    Commented Aug 15, 2018 at 19:48
  • 2
    $\begingroup$ There's some subtleties to this question. It could be interpreted in a few ways. 1) Is AdaBoost (mathematically) a special case of Gradient Boosting? (yes). 2) Was gradient boosting discovered as a way to use the AdaBoost model in more general situations (sort of). 3) Is it best to view Gradient Boosting as a generalization of AdaBoost in 2018 (No, it better to think of Gradient Boosting as a primary concept, and AdaBoost as secondary). Which was your intention (or something else entirely...)? $\endgroup$ Commented Aug 15, 2018 at 21:44
  • $\begingroup$ My main question is about '1' . If Adaboost is a generalization of Gradiant Boosting and also if using Gradiant Boosting with the right error distribution will yield the same result as adaboost. However, I am not sure what you exactly mean in number '3'. Can you explain a bit? $\endgroup$
    – ABIM
    Commented Aug 16, 2018 at 12:47


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.