# Why doesn't boosting have a really high generalization error?

AdaBoost creates a large amount of weak learners, and gives each one prediction score/weight that will be used to combine the learners during evaluation/generalization.

But isn't the point to create different learners that are stronger at different examples? If so, weighting them basd on their overall error is counterproductive. There needs to be a dynamic, state-space framework to ensembling. Is this correct?

UPDATE:
I watched the suggested video, but I still do not feel like my question has been answered. If you combine a bunch of weak learners using unchanging coefficients, the error of each individually should be pretty high because they each only target specific aspects of the example distribution.

• – Will May 7 '17 at 11:30
• Thank you for the suggestion. I watched the video, but I still do not feel like my question has been answered. If you combine a bunch of weak learners using unchanging coefficients, the error of each individually should be pretty high because they each only target specific aspects of the example distribution. – Abraham Horowitz May 7 '17 at 13:29
• Do you know gradient boosting? I feel like that algorithm is much easier to understand than AdaBoost. It's unfortunate that AdaBoost is still the first boosting algorithm that many learners encounter, it's both worse performing and more difficult to understand than gradient boosting. – Matthew Drury May 7 '17 at 19:48
• I am familiar with it. It models the residuals as opposed to modeling a re-weighted distribution like AdaBoost does. – Abraham Horowitz May 7 '17 at 20:16

$$\lambda_j = \frac{1}{2}\ln\frac{\sum_{i: h_j(x_i) y_i = 1} w_j^i}{\sum_{i: h_j(x_i) y_i = -1} w_j^i}$$
in the case of binary classification. By multiplying the example weight vector by this, the algorithm prevents the next learner at iteration $j+1$ from overfitting to residuals, but still changes the distribution enough that many iterations, the individual learners will master the whole example distribution.