Optimal feature selection

Question

I am working on classification issue. My training set contains of 10D features vectors. As a training model I am going to use Fisher or Neural Network. Here is a plot of the correlation matrix for a input variables for both signal and background.

[

My very first guess was to reject highly correlated variables. That why I removed seed_ty, seed_nLayers, seed_p.

Is it a good choice? Do you know any better way to choose the most optimal set of input parameters. How the answer for such question depend on what model do we use? I can use greedy feature selection method (looping over all possible feature subset) but It is not the smartest idea.

PS 1. The below plots presents the distribution of the input variables.

PS 2. Do you know any good book about feature selection?

Answer 1

Since you do not have that many features I would remove features whose correlation is above 75 (i.e. only the seed_ty)

Related to feature selection, I'm not really familiar with using neural networks but looping over all subsets of features is probably not a good idea due to the risk of overfitting. Particularly if you are using the classifier to assess the quality of the subset this is a really bad idea.

There are several strategies you could use. I would recommend reading Saeys, Inza and Larrañaga paper named: A review of feature selection techniques in bioinformatics, bioinformatics 23.19 (2007): 2507-2517, available here.

Personally, I would obtain individual scores for each feature using metrics such as information gain and test the model with different subsets of the top features. Do not forget to use strategies like cross-validation or bootstrap to test the model.