Appropriate statistic for describing differences between nested regression models? I have run a series of nested binary logistic and negative binomial regression models in SPSS examining the impact of an intervention on re-offending outcomes.
For example:
Model A = Individual characteristics
Model B = Individual characteristics + program characteristics
Model C = Individual characteristics + program characteristics + sentencing characteristics
Outcome A = Likelihood of re-offending (binary logistic regression)
Outcome B = Rate of re-offending (negative binomial regression)
All models are significant and I have been trying to understand the pros and cons of various statistics to describe the incremental predictive ability of each model.
Can I use a chi-square difference test to simply describe whether one model is better (eg. Model B is better than model A, but Model C is no better than Model B), or should I be using an F-Test or adjusted R-square change?
Any help would be much appreciated.
 A: Anova (using F-test) could be used for nested models:
anova(model_a, model_b, model_c)
p-value would indicate which nested model is sufficient
A: You can compare nested models with a likelihood ratio test.
However please try not to be driven only by p-values. If you have reason to believe that either program characteristics and/or sentencing characteristics should have an impact on the outcome, then include then in your model and don't do a statistical test. Also, keep in mind that depending on the goal of the modelling, you should consider whether either of the independent variables are related. For example if certain individual characteristics have a causal effect on the program or sentencing characteristics. If this is the case the program or sentencing characteristics are mediators of the individual characteristics. If your goal is inference and you want to know the total causal effect for each independent variable then you will need to fit seperate models for each characteristic, such that mediators are excluded. On the other hand, if any of the variables cause both the outcome and one of the other variables, then that variable is a confounder and should be included.
