As is known to all, feature engineering is extremely important to machine learning, however I found few materials associated with this area. I participated to several competitions in Kaggle and believe that good features may even be more important than a good classifier in some cases. Does anyone know any tutorials about feature engineering, or is this pure experience?
I would say experience -- basic ideas are:
- to fit how classifiers work; giving a geometry problem to a tree, oversized dimension to a kNN and interval data to an SVM are not a good ideas
- remove as much nonlinearities as possible; expecting that some classifier will do Fourier analysis inside is rather naive (even if, it will waste a lot of complexity there)
- make features generic to all objects so that some sampling in the chain won't knock them out
- check previous works -- often transformation used for visualisation or testing similar types of data is already tuned to uncover interesting aspects
- avoid unstable, optimizing transformations like PCA which may lead to overfitting
- experiment a lot
There is a book from O'Reilly called "Feature Engineering for Machine Learning" by Zheng et al.
I read the book and it covers different types of data (e.g categorical, text...) and describes different aspects of feature engineering that go with it. This includes things like normalization of data, feature selection, tf-idf in text.