I have several closely-related questions regarding weak learners in ensemble learning (e.g. boosting).
- This may sound dumb, but what are the benefits of using weak as opposed to strong learners? (e.g. why not boost with "strong" learning methods?)
- Is there some sort of "optimal" strength for the weak learners (e.g. while keeping all the other ensemble parameters fixed)? Is there a "sweet spot" when it comes to their strength?
- How can we measure the strength of a weak learner with respect to that of the resulting ensemble method. How do we quantitatively measure the marginal benefits of using an ensemble?
- How do we compare several weak learning algorithms to decide which one to use for a given ensemble method?
- If a given ensemble method helps weak classifiers more than strong ones, how do we tell a given classifier is already "too strong" to yield any significant gains when boosting with it?