# State of the art for bagging/model averaging? [closed]

If I estimate a collection of models predicting $Y$ by $\hat{Y}$, which methods are out there to combine these forecasts? Which methods work well/best (and why?) to improve prediction accuracy? My interest is of theoretical nature, for frequentist and bayesian approaches alike.

I am aware that this question is very open, but I want to gain an overview. Consequently, references to further sources or survey papers, book chapters, ... are also highly appreciated!

Edit: Bagging, boosting and stacking in machine learning was poposed as an answer to this question. I was very thankful for the link, and I recommend everyone interested in this question to read the post and its answers if they haven't done so already. However, the post does not answer my question: I am interested in specific methods for model averaging and bagging. The aforementioned post elaborates on the differences between the concepts of 'boosting', 'bagging', and 'stacking' rather than giving explicit different implementations. (e.g., model averaging use weights for each model. What I want to enquire with this post is the ways in which these weights can be obtained.)

## closed as too broad by Tim♦, gung♦, kjetil b halvorsen, Sven Hohenstein, JohnFeb 7 '16 at 3:29

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

• As you have already mentioned in your title I would start with looking at methods for bagging but also check out product of expert models and the Bayesian committee machine. – RustyStatistician Feb 6 '16 at 18:53
• Thanks! Do you know a good source for me to read up details? :) – Jeremias K Feb 6 '16 at 18:57
• Possible duplicate of Bagging, boosting and stacking in machine learning – Tim Feb 6 '16 at 18:59
• There's an overview paper on model averaging in the Journal of Economic Surveys, see here. It be of interest to you to check out Clements and Hendry (1999) who present a theory of forecasting non-stationary time series. Their taxonomy of forecast errors will help you on the issue of improving, or at least understanding, forecast performance. As far as I know, they've worked quite a bit on developing the theory of forecasting. See recent papers also. – Graeme Walsh Feb 6 '16 at 21:36
• Oh snap, that's a sweet overview! Have to look into it in more detail tomorrow. Hendry is always a good read, too - thank you! – Jeremias K Feb 6 '16 at 21:50