Logistic regression shows a significant predictor, but a simpler model makes the same prediction

Question

I am currently puzzled by the classification table SPSS produces for logistic regressions (procedure LOGISTIC REGRESSION). I used the block function for that procedure to produce one model with a single predictor and a second model with two variables. When I compare the two models using the classification tables, both models classify the same number of samples correctly. All numbers in the classification table match exactely. However, at the same time in the more complex model, both predictors have a beta-value with p<0.001. How can this be, if the second factor does not contribute to the overall classification accuracy? In another regression, classification accuracy even goes down for the more complex model, still all predictors have a beta-value with p<0.001.

What exactely is the link between the classification accuracy and the significance values here?

Answer 1

Classification accuracy throws out valuable information.

Consider a simplified example:

Model one predicts half the subjects to have probability 0.75 of success and the other half a probability of 0.25.

Model two predicts (for those that model 1 predicted 0.75) half at 0.9 and half at 0.6 while for those that model 1 predicted at 0.25 it predicts half at 0.1 and half at 0.4.

Now if we only look at the "Classification accuracy" by calling everyone with a prediction > 0.5 a success then the 2 models are identical, but that ignores the additional information that can be significant. But 0.4 is closer to 0.6 than it is to 0.1.

Think about if you are considering surgery and the doctor uses a logistic regression to predict your chances of having a successful surgery, would you really consider a 0.51 probability and a 0.99 probability the same (and a 0.49 probability different from a 0.51)?