You tried several models and tuned them appropriately and found the best performing model. What is the typical approach taken after this model selection step to improve the solution? E.g., is it feature engineering? If so, what are typical approaches for feature engineering? Please provide links to relevant resources.



There are many things you could change in the chain used in your prediction, in case you are not yet satisfied with your prediction performance. Options would be, for example:

  • Get more data: useful if you have too less samples for your problem, thereby a possibly bad performance.
  • Change preprocessing: useful if you e.g. were not able to reduce noise sufficiently that was recorded alongside data.
  • Change feature derivation: this is usually on of the major steps. As this is largely domain and problem dependent it is hard to give general advice. In a nutshell: you could remove features (e.g. features that don't add useful information), reduce features (e.g. create new features that replace existing ones), add features (e.g. transform data to different representation that is beneficial for your model). A book that I personally like for this purpose is Max Kuhn and Kjell Johnson (2013): "Applied Predictive Modeling." Springer New York, because it provides very practical view, explanation, and approach towards those problems IMHO. Besides this one there are many publications out there related to feature derivation (which as mentioned is largely domain specific when in comes to the details).
  • Try different models and parametrizations: this is what you already mentioned in your question.

One more thing: keep in mind that whichever approach you use, in the end need to proper evaluate it using e.g. repeated cross validation with a separate, held-back test set that is used only on the one, final model. Otherwise you can easily over-engineer your features to your problem and thereby overfit your problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.