I'm not able to see how do we use gradient descent to minimize the loss of binary classification with decision tree.
What I understood is that we first have a model (decision tree) that try to predict our y values. Our model make classification errors therefore we will use a new decision tree on our errors (observations where make wrong classifications ?) to correct our model, and we will add our new decision tree to the previous. Then we check for classification error for our newly combined model and repeat the process until we have almost no error.
Then, when do we use gradient descent and how is it used to tune our new decision trees ? When we build a new tree on our errors does it mean the observations where we made an error ?
I'm sorry if i'm not clear, i'm still a bit confused at how it works exactly.
Thank you in advance for your help.