I heard that stacking models is only worth it doing it in a Kaggle competition as everyone is dealing with the same training data, and due to time limit, feature engineering only helps a little with variance (I might be wrong). In my case, I am trying to increase my accuracy score in a binary real-world event. I've tried over hundreds of features and selected only 30 of them that gave the highest variance for CV from trial and error (uncorrelated features decided by two-tailed t-test). I think I almost exhausted all quantifiable features/information about the event, which led me to think about stacking models to further increase model accuracy.

All 30 features has increased about +0.012 in R2 accuracy relative to the average public/market score. If my goal is to reach +0.020 in R2, is it worth the effort to stack models considering the features have only increased by half of my goal of +0.020 in accuracy?

I have stacked about 5 weak models as meta features so far, delta R2 has increased by 0.001 (total relative of 0.013)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.