Is there a test that I can use for a logistic regression that is similar to the Vuong test for OLS?
2$\begingroup$ Not clear what you are asking. What are the (non-nested) models that you are trying to compare? $\endgroup$– tchakravartyMar 17, 2015 at 5:02
1$\begingroup$ Why don't you just use the Vuong test? It is not an OLS-specific test. $\endgroup$– BillMar 17, 2015 at 13:37
Indeed, there is a variety of goodness-of-fit (GoF) tests that can be applied to logistic regression as an alternative to the Vuong test (Vuong, 1989). I think that the following tests can be considered as relative equivalents in the above-mentioned context:
- Deviance statistics ($D^2$) test (also referred to simply as deviance test);
- Chi-square test; for examples, see this course notes document;
- Likelihood ratio test (LRT); for examples in R and SAS, see this page;
- GoF tests, based on so called pseudo-$R^2$ measures ($R_L^2$ and some others);
- Akaike's Information Criterion (AIC)-based tests; also BIC/DIC-based - see this discussion;
- Hosmer–Lemeshow test and its improved versions (see this critique and this answer by Prof. Frank Harrell - the comparison of GoF tests paper he referenced is available online here);
- Stukel's test - see this blog post;
- Area under the ROC curve (AUC)-based test: see this discussion here on Cross Validated.
For some theory on GoF testing for categorical models (in particular, logistic regression models), deviance and LRT, as well as additional examples in R, Stata and SAS, see this nice presentation.
It seems that there are some issues of significant importance, when considering logistic regression (LR) models. In particular, it appears to me that nested vs. saturated models, sample size, data grouping in cases with continuous predictors are specifically important in LR GoF testing context.
Hosmer D.W., & Lemeshow S. (1980). A goodness-of-fit test for the multiple logistic regression model, Communications in Statistics - Theory and Methods, 9(10), 1043-1069. doi:10.1080/03610928008827941
Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses, Econometrica, 57(2), 307–333. Retrieved from http://fisher.osu.edu/~schroeder.9/AMIS900/Vuong1989.pdf
2$\begingroup$ @Billy: My pleasure! Consider upvoting and accepting my answer, if you're satisfied with it. $\endgroup$ Mar 19, 2015 at 3:53