I'm working on a securities pricing project and have a bunch of models I'd like to stack/ensemble together. I've been using simple linear regression in R (the lm() function) so far but the results are over fitting pretty badly.

Does anyone have any suggestions for whether some other stacking method might be better or any papers/articles that describe how to stack regression models (as opposed to classification models).

up vote 7 down vote accepted

If you are experiencing over fitting you could look into regularized regression which in R can be fit using many packages such as (glmnet). There are many good tutorials for this - one is Regularization Paths for Generalized Linear Models via Coordinate Descent

You might also look at randomForest or gbm in R depending on your data.

You can try fitting many models and averaging their prediction as well (your reference to ensembles).

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.