What is the best way to know the best selection method for our logistic regression model ?

  • Forward Selection
  • Backward Elimination
  • Step-wise Selection
  • Full Model.

I am running logistic regression in SAS EG.

I have tried running each of them for the data that we have but for each method I get different results. Some of the factors are common but some are not. Basically, I just want to understand the logic behind selection of the method. So, Is there a way we can identify that for a specific data one method is better than the other based on the results that we get ? Please help.


  • 1
    $\begingroup$ This question has already been asked here (look at the linked questions too, & try the model-selection tag (the same considerations apply for all regression methods). $\endgroup$ – Scortchi - Reinstate Monica Jan 10 '14 at 21:12
  • $\begingroup$ But that thread doesn't have any correct answer which I can follow ? $\endgroup$ – learnlearn10 Jan 10 '14 at 21:14
  • 1
    $\begingroup$ It has one with seven up-votes, which should suggest it's worth reading to see what you think. (A tick next to an answer only means the person who asked it thought it useful, not that it's correct; the absence of a tick means nothing at all.) $\endgroup$ – Scortchi - Reinstate Monica Jan 10 '14 at 21:17
  • 3
    $\begingroup$ This one should also be very useful. And for the question in your penultimate sentence, the clue's in the name of our site. $\endgroup$ – Scortchi - Reinstate Monica Jan 10 '14 at 21:24
  • 3
    $\begingroup$ None of those. The first three will inevitably bias the estimates of any coefficients (variables not selected are forced to have exactly 0 effect). What is the purpose of the model? The Lasso or elastic net methods might be usefully applied in this case which applies shrinkage to size of the absolute (or absolute and squared in case of elastic net) values of the coefficients. When a coefficient is shrunk to zero it is removed from the model. I don't know is SAS has this, but the glmnet package for R will do this for you. $\endgroup$ – Gavin Simpson Jan 10 '14 at 21:55