# Feature Selection Packages in R, which do both regression and classification

I am very new to R. I am learning machine learning right now. Very sorry, if this question appears to be very basic. I am trying to find a good feature selection package in R. I went through Boruta package. It is a good package but I read that it is only useful for classification.

I want to do implement feature selection in R for regression tasks. I went through the caret package documentation but for my level, it is very difficult to understand.

Can any one please point me to a good tutorial or list any good packages or most frequently used packages in R for feature selection.

Any help would be appreciated. Thanks in advance.

• Boruta works well for regression. – user88 May 21 '14 at 8:41

You can also have a look at FSelector, varSelRF. FSelector contains multiple functions for feature selection based for example on the chi square test, on the information theory (entropy, mutual information, gain ratio,...), on the correlation between feature, consistency etc... varSelRF is a useful package for feature selection using random forests with backwards variable elimination and with importance spectrum.

• Hi FWaldner, this looks a little short for an answer. Could you consider expanding it by perhaps a sentence or two, perhaps to mention briefly what these packages do that is like or unlike the other suggestions, or even perhaps against other things? As it stands it's basically just a pair of links. – Glen_b May 29 '13 at 8:06
• It seems that varSelRF is also solely aimed at classification Random Forests rather than regression. – blmoore Jul 23 '15 at 16:15

Have you looked at the Machine Learning & Statistical Learning CRAN Task View where aside caret and Boruta quite a few other packages are mentioned?

In general, if you don't understand a specific statistical procedure regarding feature selection it might be better for you to ask a targeted question about it. The following CV link might come across as rather handy as a start: Algorithms for automatic model selection.

I suggest Rattle which has random forest feature selection (and much more). It has nice GUI and very easy to use.

GLMNET with lasso regression does feature selection.

• Can you provide detailed answer? – Learner Jan 14 '16 at 6:33

Additionally Caret package also provides feature selection methods. Here and here are a couple of tutorials on using feature selection in Caret package. Recently, a feature selection package based on the SISAL algorithm by Tikka and Hollmén is available in the CRAN.