Zero-inflated Poisson regression Vuong test: Raw, AIC- or BIC-corrected results

Question

I'm analyzing count data for a set of ten species and found that for the five species with highest detection rate, the zero-inflated poisson (ZIP) regression fits the data significantly better than the regular poisson regression using glm in R. I found this using the vuong test to compare the two models, using the following code (ZP = zone/phase combination, as the data is separated into before/after phases, control/impact zones)

summary(m2 <- zeroinfl(Squirrel ~ ZP|ZP, data = bact))
summary(p2 <- glm(Squirrel ~ ZP, family = poisson, data = bact))
vuong(p2, m2)   

However, for the next five species, the vuong test shows somewhat contradicting results for the fit of the ZIP regression and glm poisson regression. I'm wondering what the difference is between the raw, AIC-corrected BIC-corrected p-values is, and which I should be most concerned about (i.e. which p-value I should pay attention to). For example, the vuong results for sixth most common species is shown below:

Vuong Non-Nested Hypothesis Test-Statistic: 
(test-statistic is asymptotically distributed N(0,1) under the
null that the models are indistinguishible)
-------------------------------------------------------------
              Vuong z-statistic             H_A    p-value
Raw                  -0.3378267 model2 > model1    0.36775
AIC-corrected         4.5566296 model1 > model2  2.599e-06
BIC-corrected        19.4932729 model1 > model2 < 2.22e-16

If I use the Raw results, I'd conclude that the ZIP model is not better than the glm poisson model. However, using the AIC- or BIC-corrected results, I'd conclude that the ZIP model is better than the glm poisson model.

What is the difference between these three values, and how should I decide which to use as the result? Thanks!

Answer 1

I am convinced that it is incorrect to use the Vuong test -- in any of its forms -- as a test for zero-inflation. I have had a paper "The misuse of the Vuong test for non-nested models to test for zero-inflation" published that explains why. See http://cybermetrics.wlv.ac.uk/paperdata/misusevuong.pdf. I have also presented the paper at major statistics conferences and no one disagreed with me.

If you are still working on this, or zero-inflation in general, get in touch if you wish.

Answer 2

Great question, with a very un-great answer: it depends.

It depends on whether or not there actually is zero-inflation in the DGP. To say it another way, the vuong test is conditional - not a diagnosis, so there will be much to justify when it comes to your results.

The best explanation I have found is in Desmarais and Harden (2013).

First, what these are: 1) I assume you know what the Vuong test statistic (raw) is, and how it is estimated. 2) The AIC and BIC statistics are corrections on the log-likelihood estimation, because the standard (raw) statistic is biased.

So which is best? They say this (very basically): 1) If the DGP is not zero-inflated, then the BIC correction will perform the best, followed by the AIC correction, and then by the raw statistic.

2) If the DGP is zero-inflated, then each of the three tests performs similarly well.

Additional: 1) According to Desmarais and Harden, this applies just to Poisson models, and these corrections perform differently when looking at NB models. 2) Part of the reason why BIC corrections fail or succeed is because of the number of observations and (more importantly) the number of (and accuracy of) the covariates included in the inflation equation.

In other words, you'll likely need to make some assumptions about the DGP of your sample, and then choose an appropriate method based on varying the model specifications.

Simple, right?

Answer 3

This is a very interesting question I'm also searching for. Unfortunately, I couldn't find an answer yet. That's why I cannot help you with explaining the difference between Raw, AIC and BIC.

However, I can help you with your initial question, which model you should choose.

The AIC- and BIC-corrected tests are based checking model 1 > model 2 while the raw tests model 2 > model 1. So in your case, the Poisson regression should fit perfectly.