# Algorithm to select predictors in logistic regression?

I need to find a computer-driven way to come up with a model in logistic regression for an exploratory study using R. Usually I would just use the leaps or bestglm packages to find the best subset, but I have more than 300 potential predictor variables so none of them can go through the 2^300 potential models.

Are there any other computer-driven methods to select covariates for logistic regression that someone might recommend? If you could suggest the R package that implements them would be ideal.

What I have been doing so far is running bestglm in groups of 10 predictors at a time, taking note of the ones that appear in the best models and hope that I can reduce the demand on computer power. Not ideal, but I don't know how else to tackle this.

• You could do worse than to use LASSO.
– Sycorax
Feb 17, 2016 at 23:19
• I tried using the example of glmnet found here: web.stanford.edu/~hastie/glmnet/glmnet_alpha.html but, after following the steps suggested, the model selected was the intercept-only model. Just by this rudimentary approach I'm using I have found much better models so I'm not sure if LASSO would work for me here.
Feb 17, 2016 at 23:33
• If you're only getting an intercept back, that's telling you that none of the variables have a strong linear relationship with the outcome. Perhaps some interactions or basis expansions could work but basically your data is not strongly informative enough about the features that you have to justify anything more than an intercept model. The lasso is a principled approach to this problem, while what you're suggesting has serious flaws. Construct a train/test partition and compare true to predicted outcome values to see what I mean.
– Sycorax
Feb 18, 2016 at 0:15

This represents questionable statistical practice. On what statistical principle is that algorithm based? Why do you need to do "model selection" as opposed to stating a model and fitting it? Your approach will result in quite volatile models, i.e., you will find that bootstrapping the process would reveal a great deal of confusion about which predictors are selected.

• Well, these are the very early stages of a very exploratory study. Currently we have little guidance or purpose aside from "find relationships among this variable and these +300 predictors". My objective is therefore to find an algorithm that selects the best subset of predictors that best classify predicted VS actual cases in the dependent variable. If I throw all the predictors it classifies at about 20% success rate. But I have (laboriously) selected 10 that classify at around 11% success. So that makes me think if only 10 out of 300 are doing such a great job, I may just need to...
Feb 17, 2016 at 23:40
• ... look a little bit more to get me closer to the 20% classification success that I would get if I used everything.