It is generally accepted that the most important factor for successful machine learning is quality feature engineering:
Feature Engineering is the Key
At the end of the day, some machine learning projects succeed and some fail. What makes the difference? Easily the most important factor is the features used ... Feature engineering is more difficult [than learning] because it is domain-specific, while learners can be largely general purpose.
Pedro Domingos, "A Few Useful Things to Know About Machine Learning", Communications of the ACM (CACM), Vol. 55, No. 10, Pages 78-87, Oct 2012.
So I was searching for academic references on the subject of "Feature Quality Criteria", yet I had little success. Has anyone came across a credible paper or text-book that elaborates on general criteria for feature quality?
What intuitively comes to my mind is the discriminative and expressive power of features. Of course one might ask: "What do you mean by discriminative and expressive power of features? Explain them formally" ...
This is exactly why I'm searching for credible references.