I have developed a model for simple logistic regression with 1 independent ordinal variable and 4 binary independent variables. The model gives 64% correctly predicted cases, a Nagelkerke r2 of 12% and Hosmer-Lemeshow 0.125. I used forward stepwise LR where all variables were included in the equation, but if I leave out one of the variables I get 68% correctly predicted cases, nagelkerke 25% and Hosmer Lemeshow 0.450. Would you stick to the first model or in this circumstance leave out the variable for a better fit of the model...?

  • 2
    $\begingroup$ Did you do any cross-validation or the like to account for your model building? Otherwise the fit statistics you get are probably quite meaningless and any seemingly better fit may just be overfitting. $\endgroup$ – Björn Mar 14 '16 at 10:46
  • $\begingroup$ Yes I did, using crosstabs. Please see comment below from prof. Harrell; what about the nagelkerke..? $\endgroup$ – schvost Mar 14 '16 at 13:42

These issues have been dealt with at length on this site. You are making a number of errors, e.g.

  1. Hosmer-Lemeshow test is obsolete and arbitrary
  2. Fraction classified correctly is an improper accuracy scoring rule
  3. Stepwise regression without penalization is an invalid statistical technique unless highly controlled, or penalized for using the bootstrap

Formulate a model, pre-specified using subject matter considerations. Otherwise you need to use data reduction (masked to $Y$) or penalization. You didn't state the frequency of levels of $Y$.

  • $\begingroup$ Could you give some more explanations on the various errors you are pointing out? $\endgroup$ – nico Mar 14 '16 at 13:02
  • $\begingroup$ Thank you very much for your answer prof. Harrell. I have to consult a statistician and texts on these matters since I am not so experienced with these techniques; however I would kindly like to ask you if you could enclose links for further explanations..? I do not know how to employ penalization using SPSS $\endgroup$ – schvost Mar 14 '16 at 13:25
  • 1
    $\begingroup$ I suggest you consult a statistician. $\endgroup$ – Frank Harrell Mar 14 '16 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.