Vuong Test Equivalent for Logistic Regression

Question

Is there a test that I can use for a logistic regression that is similar to the Vuong test for OLS?

Answer 1

Indeed, there is a variety of goodness-of-fit (GoF) tests that can be applied to logistic regression as an alternative to the Vuong test (Vuong, 1989). I think that the following tests can be considered as relative equivalents in the above-mentioned context:

• Deviance statistics ($D^2$) test (also referred to simply as deviance test);
• Chi-square test; for examples, see this course notes document;
• Likelihood ratio test (LRT); for examples in R and SAS, see this page;
• GoF tests, based on so called pseudo-$R^2$ measures ($R_L^2$ and some others);
• Akaike's Information Criterion (AIC)-based tests; also BIC/DIC-based - see this discussion;
• Hosmer–Lemeshow test and its improved versions (see this critique and this answer by Prof. Frank Harrell - the comparison of GoF tests paper he referenced is available online here);
• Stukel's test - see this blog post;
• Area under the ROC curve (AUC)-based test: see this discussion here on Cross Validated.

For some theory on GoF testing for categorical models (in particular, logistic regression models), deviance and LRT, as well as additional examples in R, Stata and SAS, see this nice presentation.

Note

It seems that there are some issues of significant importance, when considering logistic regression (LR) models. In particular, it appears to me that nested vs. saturated models, sample size, data grouping in cases with continuous predictors are specifically important in LR GoF testing context.

References

Hosmer D.W., & Lemeshow S. (1980). A goodness-of-fit test for the multiple logistic regression model, Communications in Statistics - Theory and Methods, 9(10), 1043-1069. doi:10.1080/03610928008827941

Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses, Econometrica, 57(2), 307–333. Retrieved from http://fisher.osu.edu/~schroeder.9/AMIS900/Vuong1989.pdf