very low hosmer and lemeshow goodness of fit in logistic regression

Question

I am currently working on my masterthesis. Therefor i want to perform a logistic regression (with logit link funtction) to predict the degree of encoded registrations in gp practices (coded registrations/total registrations) with practice characteristics. I have 1800 enteries and thus a big data set. in the analysis of maximum likelihood estimates every parameter is significant and need to be retained.

if i perform an Hosmer and lemeshow goodness-of fit test the chi-square = 5002 and the p-value <.0001. The auc = 0.72. Can i concluede that my model does not fit good? How can i take this into acount or build a model that better suits? Can it be my hosmer and lemeshowtest is not good because of the big amount of participants?

proc logistic data = thesis11;
class CD_PRACT_TPE (ref='WGC');
model AMNT_CDED_DIAG / AMNT_DIAG = MEAN_AGE_CREGVR MDISCIP PERC_FEMALE_CREGVR AMNT_PC_ALL AMNT_CREGVR_ALL CD_PRACT_TPE CD_ASST / link=logit selection=stepwise lackfit;
output out = predict pred=prob;
run;

Answer 1

To summarize comments into a formal answer:

The Hosmer-Lemeshow test is a poor choice. As Frank Harrell said in a comment:

The Hosmer-Lemeshow test has been obsolete for > 20 years. Reasons for this include (1) dependence on how ties are handled when quantiles are calculated, (2) dependence on how predictions are grouped, (3) too low power, and (4) too high power, detecting trivial miscalibration with huge N.

From what you describe, issue (4) is probably at work with your data. It's better to evaluate model calibration visually over the whole range of predicted probabilities. As Dave pointed out, Harrell's rms package in R provides that via its calibrate() function; I suspect that something similar is available in SAS. The validate() function provides other tests of model quality, in particular whether the model might suffer from optimistic overfitting of the data.

If you do find inadequate calibration, you might be able to improve calibration by flexible fitting of continuous predictors, for example with regression splines. You seem to have several such predictors in your model. It's unrealistic to expect a continuous predictor's values to be exactly linearly associated with the log-odds of an outcome. A regression spline or other generalized additive model can let the data estimate the shape of the association. Your understanding of the subject matter might also suggest some interactions among predictors that could be important.

The number of cases in the minority-outcome class, not the total number of cases, primarily determines how flexibly you can try to fit the continuous predictors and how many interactions you might be able to evaluate. Do not use stepwise selection if you do extend your model. Follow the advice in Frank Harrell's Regression Modeling Strategies, in particular Chapters 2 and 4 about general strategies and Chapters 10, 11 and 12 on logistic regression.