Why doesn't boosting have a really high generalization error?

AdaBoost creates a large amount of weak learners, and gives each one prediction score/weight that will be used to combine the learners during evaluation/generalization.

But isn't the point to create different learners that are stronger at different examples? If so, weighting them basd on their overall error is counterproductive. There needs to be a dynamic, state-space framework to ensembling. Is this correct?

UPDATE:
I watched the suggested video, but I still do not feel like my question has been answered. If you combine a bunch of weak learners using unchanging coefficients, the error of each individually should be pretty high because they each only target specific aspects of the example distribution.

• – Will
Commented May 7, 2017 at 11:30
• Thank you for the suggestion. I watched the video, but I still do not feel like my question has been answered. If you combine a bunch of weak learners using unchanging coefficients, the error of each individually should be pretty high because they each only target specific aspects of the example distribution. Commented May 7, 2017 at 13:29
• Do you know gradient boosting? I feel like that algorithm is much easier to understand than AdaBoost. It's unfortunate that AdaBoost is still the first boosting algorithm that many learners encounter, it's both worse performing and more difficult to understand than gradient boosting. Commented May 7, 2017 at 19:48
• I am familiar with it. It models the residuals as opposed to modeling a re-weighted distribution like AdaBoost does. Commented May 7, 2017 at 20:16
• Are you sure that it doesn't? The point of boosting is to push an underfitting model toward lower variance. The amount you push is in the number of iterations, which is a hyperparameter. I wouldn't be suprised if Adaboost overfits in the limit of iterations (but I don't know either way, maybe it converges to a Bayes optimal classifier). Commented Feb 16, 2021 at 13:38

$$\lambda_j = \frac{1}{2}\ln\frac{\sum_{i: h_j(x_i) y_i = 1} w_j^i}{\sum_{i: h_j(x_i) y_i = -1} w_j^i}$$
in the case of binary classification. By multiplying the example weight vector by this, the algorithm prevents the next learner at iteration $j+1$ from overfitting to residuals, but still changes the distribution enough that many iterations, the individual learners will master the whole example distribution.