Extremely confused about the following:
Lets say we start out with a dumb weak learner. Since its the 0th model and hasnt learned anything yet, we have a high residual, lets say of 10,000.
We produce the next weak learner, which actually turns out to be pretty good. The residual is only 1000. And so we update the residual per the formula above to 9000 (10000-1000 = 9000).
However, when we produce the next weak learner, it turns out to be bad. The residual is 8000. And so we update the residual to 1000 (9000-8000 = 1000).
So I think you can see where my question is - how does this boosting system assign higher weight to the good weak learner of only 1000 residual when its only barely moving the "Total Residual" (outcome variable)? Even if lambda parameter scales it so that the Total Residual wont go negative, we still get a higher weighting for the bad models and a lower weighting for the good models?