Stacking is done with combining all the weak learners. What will happen if we do it with strong learners? A case of overfitting?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ No, stacking is typically done with strong learners. Bagging or boosting is usually done with weak learners. $\endgroup$– Erin LeDellCommented Oct 28, 2017 at 4:12
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You can combine any models and usually it will improve the quality of the predictions (this is done by most of the teams winning the Kaggle competitions). By combining multiple models you wouldn't rather make the solution prone to overfitting, since you will be using the joint oppinion of different models, you will benefit from the wisdom of the crowds. Different models will have, more or less, different oppinions on different cases and and they will be "averaged out". The idea of stacking weak learners is that weak learners are fast to learn, so you can easily train huge amounts of such models. It appears that a huge crowd of weak learners works remarkedly well and good enough not to waste resources on combining the computationally demanding models.
