# Comparing regression models on count data

I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression.

model.pois <- glm(Response ~ P1 + P2 +...+ P5, family=poisson(), ...)
model.pois.inter <- glm(Response ~ (P1 + P2 +...+ P5)^2, family=poisson(), ...)


Two of the models I fit with negative binomial regression.

library(MASS)
model.nb <- glm.nb(Response ~ P1 + P2 +...+ P5, ...)
model.nb.inter <- glm.nb(Response ~ (P1 + P2 +...+ P5)^2, ...)


Is there a statistical test I can use to compare these models? I've been using the AIC as a measure of the fit, but AFAIK this doesn't represent an actual test.

• You want to compare the models' fit using a statistical test, right? What kind of hypothesis would you like to test? – Firefeather Dec 16 '10 at 20:56
• @Firefeather For example, I would like to test whether the fit of model.nb.inter is significantly better than that of model.pois.inter. Yes, the AIC is lower, but how much lower constitutes significantly better? – Daniel Standage Dec 16 '10 at 21:00
• Note: the answer to this question need not actually include the AIC. – Daniel Standage Dec 16 '10 at 21:07
• I don't know the answer to this question, but I can give a start. I know you can use an $F$ test to compare model.pois against model.pois.inter (and similarly compare model.nb against model.nb.inter), but I can't guarantee that comparisons between a Poisson model and a negative binomial model would work. I wonder if an $F$ test to compare the variances of each pair would be reliable. – Firefeather Dec 16 '10 at 21:16
• @Firefeather, yes I'm aware of the need to control the familywise confidence level. Would Scheffe be more appropriate here than, say, Bonferroni? – Daniel Standage Dec 16 '10 at 21:40

Note that some statistical software packages, such as Stata, will do this all for you automatically when you fit a negative binomial model. In fact I've shamelessly cribbed much of the above from the Stata help system -- if you have Stata see help j_chibar.
I believe anova() in R can be used for this. Despite its name, it's a likelihood ratio test. Crawley in his The R Book has some examples of usage.