1
$\begingroup$

I have developed a model for simple logistic regression with 1 independent ordinal variable and 4 binary independent variables. The model gives 64% correctly predicted cases, a Nagelkerke r2 of 12% and Hosmer-Lemeshow 0.125. I used forward stepwise LR where all variables were included in the equation, but if I leave out one of the variables I get 68% correctly predicted cases, nagelkerke 25% and Hosmer Lemeshow 0.450. Would you stick to the first model or in this circumstance leave out the variable for a better fit of the model...?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Did you do any cross-validation or the like to account for your model building? Otherwise the fit statistics you get are probably quite meaningless and any seemingly better fit may just be overfitting. $\endgroup$
    – Björn
    Mar 14, 2016 at 10:46
  • $\begingroup$ Yes I did, using crosstabs. Please see comment below from prof. Harrell; what about the nagelkerke..? $\endgroup$
    – schvost
    Mar 14, 2016 at 13:42

1 Answer 1

Reset to default
3
$\begingroup$

These issues have been dealt with at length on this site. You are making a number of errors, e.g.

  1. Hosmer-Lemeshow test is obsolete and arbitrary
  2. Fraction classified correctly is an improper accuracy scoring rule
  3. Stepwise regression without penalization is an invalid statistical technique unless highly controlled, or penalized for using the bootstrap

Formulate a model, pre-specified using subject matter considerations. Otherwise you need to use data reduction (masked to $Y$) or penalization. You didn't state the frequency of levels of $Y$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Could you give some more explanations on the various errors you are pointing out? $\endgroup$
    – nico
    Mar 14, 2016 at 13:02
  • $\begingroup$ Thank you very much for your answer prof. Harrell. I have to consult a statistician and texts on these matters since I am not so experienced with these techniques; however I would kindly like to ask you if you could enclose links for further explanations..? I do not know how to employ penalization using SPSS $\endgroup$
    – schvost
    Mar 14, 2016 at 13:25
  • 1
    $\begingroup$ I suggest you consult a statistician. $\endgroup$
    – Frank Harrell
    Mar 14, 2016 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.