2
$\begingroup$

I am finding that trying to do a stepwise logistic regression is far too slow on my data set (6 hours). Is anyone aware of any faster solutions out there? Perhaps one that takes advantage of the multiple processors on my machine?

model <- glm(y ~ .) # 30 or so independent variables
# start timer
step(model, trace=1, direction="both", k=log(179188))
# end timer -- 6 hours
$\endgroup$
6
  • 1
    $\begingroup$ How long does it take to estimate the model in the first place? Have you tried direction = "forward" and "backward", they might be faster, and if they both arrive at the same solution that would work. You might try something like boosted regression instead of logistic, depending on your goals. $\endgroup$ Dec 23, 2014 at 19:48
  • 3
    $\begingroup$ If you are interested in using the logistic model for prediction (as opposed to interpreting the significance and magnitude of the coefficients) I would suggest at least exploring a flexible method from the data-mining arena such as boosted CART, random forests, or neural networks. All can adapt to non-flat response surfaces algorithmically so you don't need to run down the list of interactions. $\endgroup$
    – bsbk
    Dec 23, 2014 at 20:09
  • 1
    $\begingroup$ This question appears to be off-topic because it is about how to speed up R code. $\endgroup$ Dec 23, 2014 at 20:59
  • 5
    $\begingroup$ I would suggest you do not use stepwise selection. If you want to test hypotheses, stepwise selection will invalidate the reported p-values. If you want to build a predictive model, it will yield a model that is overfitted. There is, in essence, no good reason for using stepwise selection. If that doesn't make sense / you want to learn why, you could read my answer here: Algorithms for automatic model selection. $\endgroup$ Dec 23, 2014 at 21:02
  • 2
    $\begingroup$ @gung Thank you for the second comment: it shows why we might want to keep this question around, because it can be interpreted as an implicit request for alternatives to stepwise regression. Often enormous improvements in processing speed are achieved by using a better algorithm or procedure compared to throwing more hardware at a problem. $\endgroup$
    – whuber
    Dec 23, 2014 at 21:45

1 Answer 1

8
$\begingroup$

I gather it is the stepwise selection that is slowing you down, so you would speed up your code by skipping the stepwise selection. As it happens, I would suggest you do not use stepwise selection for other reasons as well. If you want to test hypotheses, stepwise selection will invalidate the reported $p$-values. If you want to build a predictive model, it will yield a model that is overfitted. There is, in essence, no good reason for using stepwise selection. If that doesn't make sense / you want to learn why, you could read my answer here: Algorithms for automatic model selection.

If you need to do variable selection with logistic regression for some reason, you can use the LASSO method. Some of the threads listed in this search may be helpful for you.

$\endgroup$
1
  • $\begingroup$ Thanks for the advice and the links, I will look in to using these as an alternative. $\endgroup$ Jan 27, 2015 at 0:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.