2
$\begingroup$

I have a got a fair idea about how it works in regression where each successive decision tree tries to predict the residual (negative gradient for loss function) and the predicted value gets added to the result of the previous tree. Can someone please explain how this works in case of classification? What is the residual in this case?

$\endgroup$
2
$\begingroup$

In fact, there are not too much difference between regression and classification. The only difference is the loss function. In regression, the model is trying to minimize e.g., RSME. In classification, the model is trying to minimize the logistic loss.

Details can be found here.

Regularization methods for logistic regression

$\endgroup$
  • $\begingroup$ Thanks for the clarification about the loss function. Can you please elaborate what does each successive tree try to predict? For example, as I have mentioned above in case of regression the subsequent decision trees try to predict the residuals $\endgroup$ – Abhisek Dutta Jul 9 '18 at 8:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.