# Overfit in aggregated models: boosting versus simple bagging

Let's fix a bagging setup, where several models are build independently and than somehow aggregated. It is intuitive that increasing the number of weak learners ( N ) does not lead to overfit ( in the sense that the overfitting properties do not worsen adding an arbitrary number of trees ) . This is also discussed here for random forest:

https://datascience.stackexchange.com/questions/1028/do-random-forest-overfit

I was wondering if the situation is completely the opposite when we aggregate through boosting. In the AdaBoost algorithm, for example https://en.wikipedia.org/wiki/AdaBoost , the parameters of the next weak learner are chosen so that it improves the prediction of the previous step. Does it mean that, given enough weak learners, one would (over)fit perfectly the training data-set and, a fortiori, cause bad generaliazion ?

The question refers to the (theoretical) asymtptotic behavior for large N (the number of weak learners).

• There's one misunderstanding (which is not central to your question): in bagging, we tend to employ highly flexible (strong) learners, and try to wash away their variance by averaging them all – Firebug Sep 10 '20 at 13:06
• Thank you for this remark. I guess you want to say that bagging is in general used to average low bias, high variance models, whereas boosting is thought to combine models with high bias and low variance. Is this correct ? As you noticed, this is not central to the question but thanks for the remark ! – Thomas Sep 10 '20 at 13:50
• Yes, that would be correct for most applications. Of course, deviations must exist, but I'm unaware of them. – Firebug Sep 10 '20 at 14:45