2
$\begingroup$

I have a set of data (specifics not important) where the outcome is a count, a set of predictor variables (categorical and continuous), a offset variable and a two level hierarchical structure. I first used Poisson regression to model the outcome, using a random intercept which fits the natural structure of the data.

I have been using the ratio of the sum of pearson chi squared residuals and the degrees of freedom to assess over dispersion (I know this is not a perfect measure for this). Its value was 1.94, but should be equal to 1, so I have explored other methods to account for this.

The majority of suggestions on this site say to use either a generalised poisson model (https://www.ncbi.nlm.nih.gov/pubmed/16389919) or a negative binomial distribution is the most popular suggestion. My understanding for using these is that in both of these suggestions the variance is free to vary from the mean (unlike in the poisson distribution). Therefore in my mind, the models should not be dispersed.

Despite this, my measure for over dispersion is 1.49 and 1.63 in the models respectively (should be equal to 1). My understanding is that these models should account for over dispersion, and so then either my measure is poor, or the models are in fact over dispersed.

To clarify, my questions are:

1) Can a negative binomial model (or generalised poisson) model be over dispersed?

2) If it cannot be over dispersed, does this mean my model is fitting poorly in other areas, causing the Pearson Chi Sq/DF > 1?

3) If they can be over dispersed, then what further measures could be taken to account for over dispersion.

4) There appears to be more information about this online when NOT using a multi level model. Am I naive in thinking the same methods to account for over dispersion apply when a multi level model is being used.

I can provide more detailed information about my code and data upon request, however I did not feel it necessary, as my query are more about the theory.

Improve this question
$\endgroup$
1
  • $\begingroup$ Well, if your data follows some negative binomial distribution, but there are too many zeros ("zero-inflated negative binomial") it could be said to be overdispersed relative to a negative binomial distribution. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Apr 27, 2017 at 13:12

2 Answers 2

Reset to default
5
$\begingroup$

Answer to question 1)

With count data "overdispersion" usually means relative to a Poisson distribution, but one could also say "overdispersed relative to a negative binomial distribution", meaning higher variance than the negative binomial implies. One mechanism which could produce this is that your data is generated by a mixture of two processes, one that produces zeros, another is negative binomial. That would be called a zero-inflated negative binomial model. There is certainly other possible mechanisms.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Another mechanism is when the event rates do not vary across units according to a gamma distribution (e.g. something with longer tails instead) as the negative binomial model assumes Or correspondingly that the log-rates do not vary according to a normal distribution as the most standard Poisson random effects model would assume (which is not that far off from what a negative binomial model does). Omitting key covariates from the model could be a reason for why this might happen (you might essentially e.g. get a bimodal or multimodal random effect). $\endgroup$
    – Björn
    Commented May 3, 2017 at 16:27
1
$\begingroup$

Overdispersion is not specific to counting data. In fact, any parametric model including the normal distribution that can't explain the variance in the data set is over-dispersion or under-dispersion.

To address the overdispersion, you may want to re-estimate the model with the new data set. You may also:

  • Ignore it because overdispersion isn't a problem if you don't care about the standard errors
  • Try NB-P model by William Greene (2008). An extension of NB for dispersion.
Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ To respond to your first paragraph: I understand that over dispersion is not unique to counts, and is more about when the variance function is miss specified. However what I want to know is that when the variance function is left to be general (i.e. in case of poisson, it can vary from the mean), how the model can still appear to be over dispersed? To respond to the first bullet point: I am interested in the distribution of the random effects. i.e. there are say 200 levels for the random effects, where does the prediction of each one lie. Is over dispersion likely to affect this? $\endgroup$
    – AP30
    Commented May 8, 2017 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.