I heard that stacking models is only worth it doing it in a Kaggle competition as everyone is dealing with the same training data, and due to time limit, feature engineering only helps a little with variance (I might be wrong). In my case, I am trying to increase my accuracy score in a binary real-world event. I've tried over hundreds of features and selected only 30 of them that gave the highest variance for CV from trial and error (uncorrelated features decided by two-tailed t-test). I think I almost exhausted all quantifiable features/information about the event, which led me to think about stacking models to further increase model accuracy.
All 30 features has increased about +0.012 in R2 accuracy relative to the average public/market score. If my goal is to reach +0.020 in R2, is it worth the effort to stack models considering the features have only increased by half of my goal of +0.020 in accuracy?
I have stacked about 5 weak models as meta features so far, delta R2 has increased by 0.001 (total relative of 0.013)