I have a got a fair idea about how it works in regression where each successive decision tree tries to predict the residual (negative gradient for loss function) and the predicted value gets added to the result of the previous tree. Can someone please explain how this works in case of classification? What is the residual in this case?
1 Answer
$\begingroup$
$\endgroup$
1
In fact, there are not too much difference between regression and classification. The only difference is the loss function. In regression, the model is trying to minimize e.g., RSME. In classification, the model is trying to minimize the logistic loss.
Details can be found here.
answered Jul 9, 2018 at 6:00
-
$\begingroup$ Thanks for the clarification about the loss function. Can you please elaborate what does each successive tree try to predict? For example, as I have mentioned above in case of regression the subsequent decision trees try to predict the residuals $\endgroup$– Ad94Commented Jul 9, 2018 at 8:22