Skip to main content
28 votes
Accepted

How to deal with overdispersion in Poisson regression: quasi-likelihood, negative binomial GLM, or subject-level random effect?

Poisson regression is just a GLM: People often speak of the parametric rationale for applying Poisson regression. In fact, Poisson regression is just a GLM. That means Poisson regression is justified ...
AdamO's user avatar
  • 63.7k
21 votes

Meaning of "Overdispersion" in Statistics

In a Poisson$(\lambda)$ distribution: $$ \mu=\lambda\\ \sigma^2 =\lambda\\ \implies\\ \mu=\sigma^2 $$ Consequently, when we believe we have a Poisson distribution, we expect the samples drawn from it ...
Dave's user avatar
  • 64.8k
16 votes
Accepted

Are over-dispersion tests in GLMs actually *useful*?

In principle, I actually agree that 99% of the time, it's better to just use the more flexible model. With that said, here are two and a half arguments for why you might not. (1) Less flexible means ...
Cliff AB's user avatar
  • 21.4k
14 votes

The mean and variance of Poisson distribution are equal

Just read the next two sentences: There is no way to increase the variance without increasing the mean. Unfortunately, in many data sets the variance is larger than the mean. If you model some ...
angryavian's user avatar
  • 2,328
14 votes

Are over-dispersion tests in GLMs actually *useful*?

Although this is my own question, I'm also going to post my own two-cents as an answer, so that we add to the number of perspectives on this question. The issue here is whether or not it is sensible ...
Ben's user avatar
  • 129k
12 votes
Accepted

Adding an observation level random term messes up residuals vs fitted plot. Why?

Thanks for updating your post, Charly. I played with some over-dispersed Poisson data to see the impact of adding an observation level effect in the glmer model on the plot of residual versus fitted ...
Isabella Ghement's user avatar
12 votes

Meaning of "Overdispersion" in Statistics

For many one-parameter probability distributions, the variance in the distribution is a function of the mean. When you fit data to a statistical model using these distributions, the estimator will ...
Ben's user avatar
  • 129k
10 votes
Accepted

Difference between heteroskedasticity and overdispersion

No, they are not equivalent. In fact, they are quite unrelated. Heteroskedasticity is when variance differs between "situations". For instance, in a regression task, the variance of the ...
Stephan Kolassa's user avatar
8 votes
Accepted

Can you use glmmTMB to simultaneously model offsets and zero-inflation?

tl;dr as far as I can tell at this point, ...
Ben Bolker's user avatar
  • 44.5k
8 votes
Accepted

dispersion parameter in Poisson models

That's correct! You've found out why glm doesn't use deviance/df as an estimate of dispersion: it's not a very good one. It uses the better estimate based on the ...
Thomas Lumley's user avatar
7 votes

Overdispersion tests from DHARMa and sjstats: conflicting results?

I'm the developer of DHARMa. First of all: note that results are not actually conflicting - a non-significant test doesn't mean that there is no overdispersion, it just means just that the respective ...
Florian Hartig's user avatar
7 votes
Accepted

Overdispersion in fitted generalized linear model with insignificant regression coefficients

Yes, that is true. There are only two commonly-used generalized linear model families for which the concept of overdispersion is relevant. These are Poisson regression or binomial regression when the ...
Gordon Smyth's user avatar
  • 13.2k
7 votes

Modelling count data with extreme underdispersion - what distribution?

The Conway-Maxwell-Poisson model has recently been shown to handle arbitrarily small underdispersion (see Huang 2020). For example, it is possible to have a mean of 15 and a variance of 2, say, by ...
user302932's user avatar
7 votes
Accepted

What better I use for Negative Binomial Regression with library(MASS) glm(family=negative.binomial) or glm.nb?

The negative binomial model is a generalized linear model only when the overdispersion parameter theta is known. In applications, we don't know it, and it needs to be estimated along with the other ...
Noah's user avatar
  • 35k
6 votes
Accepted

Machine Learning methods / Regression Trees for Longitudinal/Panel Count Data

The glmertree package on R-Forge (https://R-Forge.R-project.org/R/?group_id=261) extends the REEM tree approach in two directions: First, the response variable can ...
Achim Zeileis's user avatar
6 votes

Is there a test to determine whether GLM overdispersion is significant?

Yet another option would be to use a likelihood-ratio test to show that a quasipoisson GLM with overdispersion is significantly better than a regular poisson GLM without overdispersion : ...
Tom Wenseleers's user avatar
6 votes

Overdispersion in fitted generalized linear model with insignificant regression coefficients

Just to add to @GordonSmyth's answer, when you are fitting a quasipoisson or quasibinomial, the variance-covariance matrix is scaled by the dispersion value. This means the standard error of your ...
StupidWolf's user avatar
  • 5,147
5 votes

Is there a test to determine whether GLM overdispersion is significant?

Another alternative is to use the P__disp function from the msme package. The P__disp ...
Mischief_Monkey's user avatar
5 votes
Accepted

How to check overdispersion of binomial GLMMs, lme4 package

Why are the variance and Std.Dev of the random effects zero? Because the marginal variance among sites in your data is less than would be expected from a binomial variable; the variance can't be ...
Ben Bolker's user avatar
  • 44.5k
5 votes

Can a model fitted using negative binomial distribution be over dispersed?

Answer to question 1) With count data "overdispersion" usually means relative to a Poisson distribution, but one could also say "overdispersed relative to a negative binomial distribution", meaning ...
kjetil b halvorsen's user avatar
5 votes
Accepted

When do I have to check for overdispersion?

Overdispersion means more (higher) dispersion than assumed by the model so is a concept that is relative, it depends on the model used. Many (most) models do not assume anything about the ...
kjetil b halvorsen's user avatar
5 votes
Accepted

Quasi-likelihood/Quasi Poisson

What happens is that the likelihood equations depend on the distribution of Y only through the mean ($\mu$) and the variance ($V(\mu)$). Other moments of the distribution do not affect the ...
Bruna w's user avatar
  • 541
5 votes
Accepted

Investigate overdispersion in a plot for a poisson regression

One somewhat useful plot would be to plot absolute Pearson residuals against $\sqrt{\hat{y}}$ (or $\hat{y}$ or $\log(\hat{y})$...). It should look flat, and as long as the fitted mean isn't too small ...
Glen_b's user avatar
  • 286k
5 votes

Why fitting a Poisson GLM in an over dispersed dataset underestimate the standard error of the regression parameter?

As a result of the Poisson probability model, the variance and the mean of that distribution are the same. This fact entails that whenever this condition is not met, there will be a mismatch between ...
Guilherme Marthe's user avatar
5 votes

Running a Negative Binomial Regression with Overdispersion

I am less experienced with NB regression, but can offer my insight if others don't chime in. To your questions: I am testing the number of days people go running in a week (dependent variable) ...
Shawn Hemelstrand's user avatar
5 votes

How much dispersion is too much for quasipoisson regression?

Definition of overdispersion In the linked question, the variance is about $206 \times$ the mean of the response variable, but this number is irrelevant to the amount of overdispersion. Overdispersion ...
Frans Rodenburg's user avatar
5 votes
Accepted

Validity of AIC When Comparing Models with Varying Dispersion Parameters

Technically speaking, Akaike's information criterion $$\mathrm{AIC} = 2p - 2\log(\mathcal{L})$$ is undefined for quasi-models, because they are not fitted by likelihood, but by quasilikelihood.$^\...
Frans Rodenburg's user avatar
4 votes
Accepted

Is Quasi-Poisson the same thing as fitting a Poisson GEE model?

Quasi-poisson GLM is a special case of Poisson GEE. The specification of GEE (copied from wikipedia) is that $$U(\beta)=\sum_i \frac{\partial\mu_{ij}}{\partial \beta_k} V_i(Y_i-\mu_i(\beta))$$ ...
JDL's user avatar
  • 1,404
4 votes

Difference between Quasi-Poisson and Sandwich Covariance

From a purely applied perspective, my experience is that the difference between these methods is typically not huge, leading to qualitatively the same conclusions (see Table 2 in the vignette you ...
Achim Zeileis's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible