# Boosted regression trees update of residuals

In my book on statistical learning, an algorithm for boosting for regression trees is described. They have the main step of the algorithm as:

$\hat{f}(x) \leftarrow f(x) + \lambda \hat{f}^b(x)$,

where $\lambda$ is the shrinkage parameter, and $\hat{f}^{b}$ represents the recently computed boosted tree.

My question is how to derive the following rule for the update of the residuals:

$r_i \leftarrow r_i - \lambda \hat{f}^{b}(x)$

I've been stuck on this for quite a while. It seems simple, but I cannot get it.

• I have been under the impression, perhaps falsely, that the residual for a given iteration is just $y - \hat{f}(x)$. In other words, the residual for the current ensemble of boosted trees. Are you sure that it's more complicated than that? – generic_user Aug 20 '18 at 17:43
• Hi. Yes, according to my textbook which is "Introduction to Statistical Learning" by James and Witten. It is Algorithm 8.2 in the textbook on page 322. – Thomas Moore Aug 20 '18 at 18:06
• OK, well maybe that explains why my hand-coded attempt at replicating it didn't work so well! – generic_user Aug 20 '18 at 18:10
• If $r_i = y_i - \hat{y}_i$ then $\text{new} r_i = y_i - \hat{f} = y_i - (f + \lambda f^b) = \text{old} r_i - \lambda f^b$ no? – Fabian Werner Aug 20 '18 at 18:50