Best methods of feature selection for nonparametric regression A newbie question here. I am currently performing a nonparametric regression using the np package in R. I have 7 features and using a brute force approach I identified the best 3. But, soon I will have many more than 7 features!
My question is what are the current best methods for feature selection for nonparametric regression. And which if any packages implement the methods. Thank you.
 A: Unless identification of the most relevant variables is a key aim of the analysis, it is often better not to do any feature selection at all and use regularisation to prevent over-fitting.  Feature selection is a tricky procedure and it is all too easy to over-fit the feature selection criterion as there are many degrees of freedom.  LASSO and elastic net are a good compromise, the achieve sparsity via regularisation rather than via direct feature selection, so they are less prone to that particular form of over-fitting.
A: Lasso is indeed a good one. Simple things like starting with none, and adding them one by one sorted on 'usefullness' (via cross-validation) do also work quite well in practice. 
This is sometimes called stagewise feedforward selection.
Note that the subset selection problem is fairly independent on the type of classification / regression. It's just that nonparametric methods can be slow and therefore require more intelligent methods of selection.
The book 'The elements of statistical learning' from T. Hastie gives a nice overview.
