I recently read this blog and it has many ideas for ensembling various models. I created three models for my training data, random forest model, SVM model and a KNN model. However when I use linear method to stack these ensemble methods together the error I get is lower than the RF model only about 40% of the time.

The linear method that I employ for stacking is a plain one: determining coefficients of each model and minimizing their least squares.

Am I missing something? This results in a "best case" scenario less than half the time, so I would be better off just using the randomforest model.

Secondly, What exactly are meta features as described in this paper? How do I extract them for any data and successfully use them to build a lower error ensemble that individual models themselves?

  • $\begingroup$ I'm not sure you are doing the right ensemble without detail information. Did you use the cross validation in three models? Are you using the prediction of three models as features to train linear regression? $\endgroup$ – wolfe Jul 28 '17 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.