1
$\begingroup$

should GBDT use the same loss functions for each weak learner? The answer seems yes. I understand as the principle of GBDT is gradient descent. So if we use different loss functions, the gradient descent will not make sense. Am I right? Just want a confirmation.

$\endgroup$
2
$\begingroup$

I assume GBDT means gradient boosting with decision trees. GBDT is not a popular acronym. If you use full words may be more people will understand.

To your question.

loss function is for the whole model not for each tree / weak learner. You are correct, having one loss function and one gradient is the way to make the gradient decent work.

Each weak learner will slightly improve the loss, but not trying to search and do heavy optimization on it.

My old answer has a good demo on details for decision stump boosting

How does linear base learner works in boosting? And how does it works in the xgboost library?

Is a decision stump a linear model?

In the demo, we are approximating the quadratic function with boosted stumps. So the loss is the "differences between these two isocontour".

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.