I have a data set of around 5000 features. For that data I first used Chi Square test for feature selection; after that, I got around 1500 variables which showed significance relationship with the response variable.
Now I need to fit logistic regression on that. I am using glmulti package for R (glmulti package provides efficient subset selection for vlm) but it can use only 30 features at a time, else its performance goes down as the number of rows in my dataset is around 20000.
Is there any other approach or techniques to solve the above problems? If I go by the above method it will take too much time to fit the model.