# Steps in gradient boosting algorithm

Can some one please explain the 3rd step 2(c) in the below gradient boosting algorithm. I was under the impression, that the 2(c) computation is nothing but the mean of the corresponding terminal node (average of all the target values in the node- average of $$r_{im}$$, since $$r_{im}$$ is the target).

Also, isn't $$f_{m-1}(x_i)$$ assigned to $$\gamma$$ (a constant, in step 1) ? Not sure, why we are adding $$f_{m-1}(x_i)$$ to $$\gamma$$ which is like $$2*\gamma$$ in 2(c). Why are we using $$f_{m-1}(x_i)$$ and $$\gamma$$ and $$L$$, instead of mean of $$r_{im}$$ of the node, in step 2(c)

• Can you please check that the latexing of the Maths in your post has not changed your question? Also, did you mean to type a question in the end of the first paragraph? – usεr11852 Mar 3 '20 at 22:43
• Thank you for the latex, no not intending for any question in first paragraph – tjt Mar 4 '20 at 4:15

1. In step 1 we cannot assign $$f_{m-1}(x_i)$$ to anything as we have yet to estimate $$f$$. We just set it as the mean of the $$y_i$$ across all the samples (as we have yet to define any regions $$R_j$$).
• Your intuition about step 2(c) is correct. Just note that because of the existence of $$f_{m-1}(x_i)$$, $$\gamma_{jm}$$ will be expressed in terms of the residuals $$r_{im}$$. We effectively do use the "mean of $$r_{im}$$ of the node" as you say. Just that mean of that residuals is within the region $$j$$.
• We are not adding $$2\gamma$$ because $$f_{m−1}(x_i)$$ is extremely unlikely to equal $$\gamma$$; $$\gamma$$ is effectively in the scale of $$r$$ as it is "a mean of residuals", $$f_{m−1}(x_i)$$ on the other hand is on the scale of the response variable $$y$$.
2. Note that in step 2(d) we are adding $$\gamma_{jm}$$ in our estimates of $$x_i$$ only if $$x_i$$ is within $$R_{jm}$$. Notice that the summation is across the $$J$$ regions.
• I am happy to help. Please see the additional point particular to your query about "$2\gamma$. I also now clarify that we are talking about a "mean of (that) residuals". I hope these answer your question. – usεr11852 Mar 5 '20 at 0:27
• 1. At step 1 we have $m=0$. 2. As mentioned within the iterations $\gamma$ is the mean of the residuals $r$ in each corresponding region. 3. The RHS of 2(c) has $L$ which is directly associated with $r$. – usεr11852 Mar 5 '20 at 1:17