17
$\begingroup$

What is the appropriate strategy for deciding which model to use with count data? I have count data that i need to model as a multilevel model and it was recommended to me (on this site) that the best way to do so this is through bugs or MCMCglmm. However i am still trying to learn about bayesian statistics, and i thought i should first try to fit my data as generalized linear models and ignore the nested structure of the data (just so i can get a vague idea of what to expect).

About 70% of the data are 0 and the ratio of variance to the mean is 33. So the data is quite over-dispersed.

After trying a number of different options (including poisson, negative binomial, quasi and zero inflated model) i see very little consistency in the results (varying from everything is significant to nothing is significant).

How can i go about making an informed decision about which type of model to choose based on the 0 inflation and over-dispersion? For instance, how can i infer that quasi-poisson is more appropriate than negative binomial (or vise versa) and how can i know that using either has dealt adequately (or not) with the excess zeros? Similarly, how do i evaluate that there is no more over-dispersion if a zero-inflated model is used? or how should i decide between a zero inflated poisson and a zero inflated negative binomial?

Improve this question
$\endgroup$

4 Answers 4

Reset to default
10
$\begingroup$

You can always compare count models by looking at their predictions (preferrably on a hold out set). J. Scott Long discusses this graphically (plotting the predicted values against actuals). His text book here describes in details but you can also look at 6.4 on this document.

You can compare models using AIC or BIC and there is also a test called Voung test that I am not terribly familiar with but can compare zero inflated to non nested models. Here is a Sas paper describing it briefly on page 10 to get you started. It also is implmented in R posting

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the advice. I will definitely try to examine the predictions before deciding on the model $\endgroup$
    – George Michaelides
    Feb 25, 2011 at 14:41
5
$\begingroup$

A couple things to add to what B_Miner said:

1) You wrote that the models varied from "everything significant" to "nothing significant" but this is not a good way to compare models. Look, instead, at predicted values (as B_miner suggested) and effect sizes.

2) If 70% of the data are 0, I can't imagine that a model without 0 inflation is appropriate.

3) Even if you don't want to go Bayesian, you can use GLMMs in SAS (PROC GLIMMIX or NLMIXED) and in R (various packages). Ignoring the nested nature may mess everything up.

4) In general, deciding on which model is best is an art, not a science. There are statistics to use, but they are a guide to judgment. Just looking at what you wrote, I would say a ZINB model looks right

Improve this answer
$\endgroup$
1
  • $\begingroup$ The intention is that i will eventually try to model this using Bayesian, but i was trying to understand how i can make a decision before fitting the models. If there is a possibility that ignoring the nested nature of the data messes things up, them i will try them GLMMs first. The only package for R that i am aware of that can do multilevel ZINB is glmmADMB. Would you recommend any other packages? $\endgroup$
    – George Michaelides
    Feb 25, 2011 at 14:48
5
$\begingroup$

My understanding is that zero-inflated distributions should be used when there is a rationale for certain items to produce counts of zeroes versus any other count. In other words, a zero-inflated distribution should be used if the zeroes are produced by a separate process than the one producing the other counts. If you have no rationale for this, given the overdispersion in your sample, I suggest using a negative binomial distribution because it accurately represents the abundance of zeroes and it represents unobserved heterogeneity by freely estimating this parameter. As mentioned above, Scott Long's book is a great reference.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your answer. Indeed, i started thinking about whether different items could produce the 0s versus any other count and I actually think that there are a couple of my variables that would only explain 0s vs any other count. So probably i should at least try ZINB first to see if my these variables work the way i would expect them to work. $\endgroup$
    – George Michaelides
    Feb 25, 2011 at 14:59
4
$\begingroup$

absolutely agreed to what Matt said, first you have to think about the background of the data... It doesn't make any sense to fit ZI models, when there are no Zero generating triggers in the population! The advantage of NB models are that they can display unobserved heterogenity in a gamma distributed random variable. Techniqually: The main reasons for overdispersion are unobs Heterogenity and Zero inflation. I do not believe that your fit is bad. Btw to get the goodness of fit you should always compare the Deviance with the degrees of freedom of your model. If the Deviance D is higher than n-(p+1) (this is df) than you should search a better model. Although there are mostly no better models than ZINB to get rid of overdispersion.

if you want to fit a ZINB with R, get the package pscl and try to use the command zeroinfl(<model>, dist=negative). For further information see ?zeroinfl after loading the required package!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.