# Comparing regression models on count data

I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression.

model.pois <- glm(Response ~ P1 + P2 +...+ P5, family=poisson(), ...)
model.pois.inter <- glm(Response ~ (P1 + P2 +...+ P5)^2, family=poisson(), ...)


Two of the models I fit with negative binomial regression.

library(MASS)
model.nb <- glm.nb(Response ~ P1 + P2 +...+ P5, ...)
model.nb.inter <- glm.nb(Response ~ (P1 + P2 +...+ P5)^2, ...)


Is there a statistical test I can use to compare these models? I've been using the AIC as a measure of the fit, but AFAIK this doesn't represent an actual test.

• You want to compare the models' fit using a statistical test, right? What kind of hypothesis would you like to test? Commented Dec 16, 2010 at 20:56
• @Firefeather For example, I would like to test whether the fit of model.nb.inter is significantly better than that of model.pois.inter. Yes, the AIC is lower, but how much lower constitutes significantly better? Commented Dec 16, 2010 at 21:00
• Note: the answer to this question need not actually include the AIC. Commented Dec 16, 2010 at 21:07
• I don't know the answer to this question, but I can give a start. I know you can use an $F$ test to compare model.pois against model.pois.inter (and similarly compare model.nb against model.nb.inter), but I can't guarantee that comparisons between a Poisson model and a negative binomial model would work. I wonder if an $F$ test to compare the variances of each pair would be reliable. Commented Dec 16, 2010 at 21:16
• @Firefeather, yes I'm aware of the need to control the familywise confidence level. Would Scheffe be more appropriate here than, say, Bonferroni? Commented Dec 16, 2010 at 21:40

Note that some statistical software packages, such as Stata, will do this all for you automatically when you fit a negative binomial model. In fact I've shamelessly cribbed much of the above from the Stata help system -- if you have Stata see help j_chibar.
I believe anova() in R can be used for this. Despite its name, it's a likelihood ratio test. Crawley in his The R Book has some examples of usage.