0
$\begingroup$

In gradient boosting we fit a regression tree to the negative gradient of the loss function. I understand that in case the loss function is the mean square error the two metrics are similar but in other cases for example the exponential loss function they differ.

Why we fit the regression tree to the negative gradient instead of residuals?

$\endgroup$
  • $\begingroup$ I have read somewhere that the residuals are defined only at the points where we have data and we use the negative gradient instead to introduce a structure to the model but i can not really understand the concept. $\endgroup$ – gnikol Mar 1 '18 at 19:11
  • $\begingroup$ Well, it is called "gradient" boosting for a reason! And there's no point in having a loss function other than MSE if you're going to ignore it while fitting and just fit to the residuals, which is gradient boosting with an MSE loss function. $\endgroup$ – jbowman Mar 1 '18 at 21:34
  • $\begingroup$ What do you mean by "there's no point in having a loss function other than MSE if you're going to ignore it while fitting and just fit to the residuals"?? $\endgroup$ – gnikol Mar 1 '18 at 22:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.