Meaning of the Boosting algorithm for Regression Trees I have a problem with understanding the concept of the Boosting Algorithm.
1. Set ˆ f(x) = 0 and ri = yi for all i in the training set.
2. For b = 1, 2, . . .,B, repeat:
(a) Fit a tree ˆ fb with d splits (d+1 terminal nodes) to the training
data (X, r).
(b) Update ˆ f by adding in a shrunken version of the new tree:
ˆ f(x) ← ˆ f(x) + λ ˆ fb(x). 
(c) Update the residuals,
ri ← ri − λ ˆ fb(xi). 
3. Output the boosted model,
ˆ f(x) =  sum from b=1 to B :   λ ˆ fb(x). 



*

*Why is this method tagged as "learn slowly"? where is the learning taking place?

*why is lambda's magnitude relates to "how fast will the method learn"?

*what is the meaning when d=1? it's called a "stump" but I didn't exactly understood what it means...


Thanks for answer
 A: 1) The algorithm is said to "learn slowly" because the value of the learning rate $\lambda$ is usually set to some small number, values of $.01$ or even $.001$ are common.  The boosting algorithm is really trying to maximize (or minimize) a loss function.  Setting $\lambda$ to a small value allows the algorithm to adjust its course more often, and hopefully trace a more accurate path towards the optimum.  Think of walking to the top of a hill, if you're a regular sized person you will not have much trouble getting to the exact top, but if you are twice the size of the hill you may easily step right over it.
This idea comes from solving optimization problems with gradient ascent (for maximization, it's descent for minimization).  If you want to find the maximum of a function $f$, a good way is to walk in the direction of the gradient (which is the direction that the function increases at the fastest rate):
$$ x_{i+1} \leftarrow x_i + \nabla f(x_i) $$
It is observed in many problems that this overshoots, and then spends much time stepping back and forth over, the maximum.  To combat this effect, it is common to use a small learning rate to temper the size of the steps:
$$ x_{i+1} \leftarrow x_i + \lambda \nabla f(x_i) $$
2) The smaller you make $\lambda$ the smaller steps you take, and the longer you will take to reach the top of the hill (optimal model).
3) A stump is a tree with two terminal nodes, or, saying the same thing, one split.  This is a rather stretched analogy with a "tree stump", what is leftover when you cut a tree down to the ground.
