# Best algorithm for Feature Selection in classification use case

I am implementing some machine learning algorithms on a large data set (90K rows) with 274 different variables. I have to carry out Logistic Regression and Random Forest for this data set. meanwhile, I want to carry out feature selection to reduce the number of those variables drastically.

What would be an effective feature selection algorithm (in R) for classification use case? Thanks, Aman

• This does not seem like horribly large. Try a LASSO approach (check the glmnet package for an authoritative R implementation) for starters (or a Elastic Net). This should give you good mileage. Mar 23 '16 at 17:54
• Importance and feature selection is still active. There is no "unilateral best answer", yet. One library to look at is the "boruta" library. Another (mentioned below) is caret. Another is cubist. Mar 23 '16 at 19:42
• What are your intended aim with feature selection? (increased predictiveness, simpler modeling, etc.) The sentence "I want to carry out feature selection to reduce the number of those variables: drastically." only says you want to remove features because you want to remove a lot of features. Your data set is quite tall(n>>p) so feature selection is not necessarily needed. Mar 24 '16 at 12:57

You could explore recursive feature selection in the caret library in R, if you have available resources. Another factor to consider is the frequency of training of your models.

This approach can be resource intensive, so remember to run in parallel. You set the function to use the randomForest algorithm which then recursively runs through all your features in sequential training sessions. By doing this it will select the appropriate amount of variables that should be use. As well as the best Random Forest model based on a evaluation metric like kappa.

Go read up on the applicability of the rfe function here: Caret on Github

You should take a look at Maximum relevancy minimum redundancy (MRMR) algorithm

It is a very simple greedy algorithm, implementing it in R should be a breeze

• you should mention that mrmr is not applicable if you have unordered categorical variable in your dataset Apr 13 '19 at 19:12