It is not fully clear to me if Gradient Boosted Trees are boosting on the residuals of each stage or on the complete training set?

From the Wikipedia page it seems to me that the Overal "Algorithm" as explained for Gradient Boosting is boosting on the residuals but when the article explains "Gradient tree boosting" - residuals are never mentioned - emphasis is given to gamma-i which weights each individual weak learner added to the model.

So what is true now?

  1. Is Gradient Boosted Tree boosting on the residuals or on the complete training set?

  2. Gradient Boosted Tree weighting individual training examples as for example explained in AdaBoost?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.