12
votes
$\begingroup$

I am very new to R. I am learning machine learning right now. Very sorry, if this question appears to be very basic. I am trying to find a good feature selection package in R. I went through Boruta package. It is a good package but I read that it is only useful for classification.

I want to do implement feature selection in R for regression tasks. I went through the caret package documentation but for my level, it is very difficult to understand.

Can any one please point me to a good tutorial or list any good packages or most frequently used packages in R for feature selection.

Any help would be appreciated. Thanks in advance.

$\endgroup$
1
  • 3
    $\begingroup$ Boruta works well for regression. $\endgroup$
    – user88
    Commented May 21, 2014 at 8:41

5 Answers 5

Reset to default
13
votes
$\begingroup$

You can also have a look at FSelector, varSelRF. FSelector contains multiple functions for feature selection based for example on the chi square test, on the information theory (entropy, mutual information, gain ratio,...), on the correlation between feature, consistency etc... varSelRF is a useful package for feature selection using random forests with backwards variable elimination and with importance spectrum.

$\endgroup$
2
  • 2
    $\begingroup$ Hi FWaldner, this looks a little short for an answer. Could you consider expanding it by perhaps a sentence or two, perhaps to mention briefly what these packages do that is like or unlike the other suggestions, or even perhaps against other things? As it stands it's basically just a pair of links. $\endgroup$
    – Glen_b
    Commented May 29, 2013 at 8:06
  • $\begingroup$ It seems that varSelRF is also solely aimed at classification Random Forests rather than regression. $\endgroup$
    – blmoore
    Commented Jul 23, 2015 at 16:15
7
votes
$\begingroup$

Have you looked at the Machine Learning & Statistical Learning CRAN Task View where aside caret and Boruta quite a few other packages are mentioned?

In general, if you don't understand a specific statistical procedure regarding feature selection it might be better for you to ask a targeted question about it. The following CV link might come across as rather handy as a start: Algorithms for automatic model selection.

$\endgroup$
0
3
votes
$\begingroup$

I suggest Rattle which has random forest feature selection (and much more). It has nice GUI and very easy to use.

$\endgroup$
0
1
vote
$\begingroup$

GLMNET with lasso regression does feature selection.

$\endgroup$
1
  • 1
    $\begingroup$ Can you provide detailed answer? $\endgroup$
    – Learner
    Commented Jan 14, 2016 at 6:33
1
vote
$\begingroup$

Additionally Caret package also provides feature selection methods. Here and here are a couple of tutorials on using feature selection in Caret package. Recently, a feature selection package based on the SISAL algorithm by Tikka and Hollmén is available in the CRAN.

$\endgroup$
1
  • $\begingroup$ Link to tutorial is dead. However, this link could be helpful. $\endgroup$
    – Ekaba Bisong
    Commented Nov 25, 2016 at 7:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.