I know the basic overview of how gradient boosting trees work but i am finding it hard to figure out the use of gradient in gradient boosting. My questions may seem stupid but it would be great if anyone can help me out !. So my question is

P.S : It would be even more helpful if someone can explain me with less math

• your question of "what is a gradient" is really broad. Have you tried reading any tutorials or articles about it, such as this one by Khan Academy? If so, can you make your question more specific? – jld Apr 4 '18 at 23:51
• @Chaconne Thanks for the link. I have edited the question now . I am more interested in " what is the use of gradient in gradient boosting and why is it used"? – karthik prasanna Apr 5 '18 at 1:29
• you should definitely give this a read: blog.kaggle.com/2017/01/23/… – jld Apr 5 '18 at 6:41

Suppose you have a true value $y$ and a predicted value $\hat{y}$. The predicted value is constructed from some existing trees. Then you are trying to construct the next tree which gives a prediction $z$. Then your final prediction will be $\hat{y}+z$. The correct choice of $z$ is $z = y - \hat{y}$. Therefore, you are now constructing trees to predict $y - \hat{y}$.
It turns out this is a special case of gradient boosting when your loss function is $L = \frac{1}{2} (y - \hat{y})^2$, and your prediction target for this new tree is the gradient of this loss function as $y - \hat{y} = - \frac{\partial L}{\partial \hat{y}}$.
In a more formal definition, if you already have a prediction $\hat{y}$, and you are trying to add a new prediction $z$ from a new tree to it, then the loss function can be expanded by Taylor's expansion near $\hat{y}$ as $$L = L_0 + \frac{\partial L}{\partial \hat{y}} z$$ With the spirit of gradient descent, we want $z$ to be along negative gradient direction, hence $z \sim - \frac{\partial L}{\partial \hat{y}}$.