# In Gradient Boosting Tree, why do we fit the tree on the residuals and not on the sum of the previous function and the residuals?

In the Gradient Boosting Tree algorithm, as described in https://en.wikipedia.org/wiki/Gradient_boosting#Gradient_tree_boosting, we update the previous model $$F_m$$ by adding the results $$h_m$$ of the decision tree applied to the residuals multiplied by a $$\gamma$$ coefficient. Why don't we fit the decision tree on $$F_m - \eta r_m$$, with $$r_m$$ the residuals and $$\eta$$ a learning rate ? The results would give us a new function $$F_{m+1}$$.

• You are correct that these results would give us a new function $F_{m+1}$. $\gamma$ is used to make control the relative step among all tree regions. It is actually post-multiplied by $\nu$ in most cases. Side-note: Wikipedia is great but because it explore many different implementations, it is a bit "jumpy" and hard to follow. I suggest, first to focus on single source expositions (e.g. the Hastie el al. ESL book, Chapt. 10, or Shapiro and Freund's early Adaboost papers) and complement the parts that are a bit fuzzy with Wikipedia rather than vice versa. – usεr11852 says Reinstate Monic Dec 8 '18 at 9:17