As I understand the idea of gradient boosting in the (m+1)-th step we take the partial derivatives of the loss with respect to our new parameters $f^{[m]}(x^{(i)})$: $\tilde{y}^{(i)}=-\frac{\partial (\sum_{i=1}^nL(y^{(i)}, f^{[m]}(x^{(i)})))}{\partial f^{[m]}(x^{(i)})}$
and afterwards approximate this gradient step with a function $f_\theta$ from our hypothesis space that comes close to this gradient step, i.e. $\tilde{f}= \text{argmin}_{f \in H}\sum_{i=1}^n\left[\tilde{y}^{(i)}- f_{\theta}(x^{(i)}) \right] ^2$
we then multiply $\tilde{f}$ by some constant (depending on the learning rate) to get $f^{[m+1]}$
Now my question is about the scenario in which our loss is not $0.5(y-f(x))^2$ but some other arbitrary loss function $L^*$. Then $\tilde{y}^{(i)} \neq (y^{(i)}-f^{[m]}(x^{(i)}))$. According to algorithm 1 in the Friedman paper (page 1193) we would then still minimize $\sum_{i=1}^n\left[\tilde{y}^{(i)}- f_{\theta}(x^{(i)}) \right] ^2$ in order to get our new function.
My question is why we would not simply minimize $\sum_{i=1}^nL(y^{(i)}-f^{[m]}(x^{(i)}), f_\theta(x^{(i)}))$
Is this only because the approach as in algorithm 1 by Friedman is cheaper or is there another reason?
