I have developed a model for simple logistic regression with 1 independent ordinal variable and 4 binary independent variables. The model gives 64% correctly predicted cases, a Nagelkerke r2 of 12% and Hosmer-Lemeshow 0.125. I used forward stepwise LR where all variables were included in the equation, but if I leave out one of the variables I get 68% correctly predicted cases, nagelkerke 25% and Hosmer Lemeshow 0.450. Would you stick to the first model or in this circumstance leave out the variable for a better fit of the model...?
These issues have been dealt with at length on this site. You are making a number of errors, e.g.
- Hosmer-Lemeshow test is obsolete and arbitrary
- Fraction classified correctly is an improper accuracy scoring rule
- Stepwise regression without penalization is an invalid statistical technique unless highly controlled, or penalized for using the bootstrap
Formulate a model, pre-specified using subject matter considerations. Otherwise you need to use data reduction (masked to $Y$) or penalization. You didn't state the frequency of levels of $Y$.