3
$\begingroup$

There is more than one solution for the problem of overdispersed count data. One is to use a quasipoisson model. One is to use a negative binomial model. One is to use a mixed-level model with subject-level random intercepts. Is there a rational and non-arbitrary way to choose among these? I ask because of a specific behavior I discovered for some overdispersed data. I have laid out all the details for this behavior in the following Kaggle notebook:

https://www.kaggle.com/bryanmaloney/dealing-with-overdispersion-which-model-to-use/

I am adding an update to summarize information given so far.

The quasipoisson, according to the linked thread, has the trait of making fewer assumptions than do likelihood-based methods. It can be somewhat emulated by an "NB1 parameterization" of the negative binomial, but glm.nb uses the "NB2 parameterization.

Negative binomial glm and poisson glmer with subject-level intercepts are the same types of model, if you ignore random intercepts of the glmer and only look at the fixed effects estimates. The major difference is that negative binomial assumes a gamma distribution of individual effects and glmer assumes a gaussian distribution.

So it comes down to how much you know about your data, or such is my guess. Do you know enough about your data to assume that the subject-level effects that contribute to overdispersion can be (at least roughly) parameterized with either a gaussian or gamma approximation? If no, then use quasipoisson. Assign to negative binomial or glmer based on your willingnesss to presume about the specific distribution. If you don't know these things or at least have good reason to presume these things, use quasipoisson.

Is this a good summary of useful principles?

$\endgroup$
  • 2
    $\begingroup$ Note: The negative binomial model is a mixed model with a subject-level random effect (the rates are assumed to vary across subjects according to a gamma distribution), just with the random effects integrated out (which makes not difference for many questions - i.e. whenever you are not really interested in the random effects). $\endgroup$ – Björn Jan 25 '18 at 16:52
  • $\begingroup$ What do you mean by "rational and non-arbitrary?" I ask because you seem dissatisfied with the methods you mention, and it would help for your audience to understand you particular needs $\endgroup$ – Alexis Jan 25 '18 at 16:56
  • $\begingroup$ My particular needs are that I am getting three different outcomes. What choice would not amount to "whatever" or "wing that mutha" or "whatever floats your boat"? $\endgroup$ – Bryan Jan 25 '18 at 16:59
  • 2
    $\begingroup$ Okay, so the actual difference between the negative binomial and the subject-level mixed model is that nb presumes a gamma distribution for individual effects and the subject-level mixed model presumes a gaussian distribution. That does explain why nb and mixed-level outcomes resemble each other marginally more than either resembles quasipoisson. Then what would be a basis for selecting between these two assumptions? $\endgroup$ – Bryan Jan 25 '18 at 17:01
  • 2
    $\begingroup$ One criterion, for that particular choice, would be whether you think the individual effects are likely to have significant positive skew or not. If you think they are not far from symmetric, then the Gaussian / subject-level mixed model will work; if you think there can be a few individual effects that are large and positive relative to the bulk of individual effects, the Gamma assumption of the NB model will work better. $\endgroup$ – jbowman Jan 25 '18 at 19:17
0
$\begingroup$

I must credit Björn and jbowman with the solution. The answer, as far as I can tell, lies in your data's structure. That is, if you have reason to believe that the individual level effects have a gaussian distribution, use a poisson mixed-level generalized linear model (glmer) with subject-level random intercepts. If you reason to believe that the individual level effects are gamma distributed, use a negative binomial model. If you have no basis whereupon to presume either of these, use the model with the least assumptions, which would be quasipoisson.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.