# How much dispersion is too much for quasipoisson regression?

Quasipoisson regression goes beyond standard poisson regression in taking into account overdispersion (whereby the dependent variable's variance is much greater than its mean). This is explained at this article.

But how much dispersion is so much that it should rule out even the use of quasipoisson regression?

In one highly upvoted answer on this site, quasipoisson is used even when the variance is 206 times the mean. Is this sound?

## Definition of overdispersion

In the linked question, the variance is about $$206 \times$$ the mean of the response variable, but this number is irrelevant to the amount of overdispersion.

Overdispersion means that the model underestimates the variance in the outcome at any given value of the explanatory variable.

In other words, overdispersion does not mean $$\frac{\mathbb{E}\bigl[ (Y - \mathbb{E}\left[Y\right])^2\bigr]}{\mathbb{E}\left[ Y \right]} > 1,$$ but instead, it means $$\frac{\mathbb{E}\Bigl[ \bigl(Y - \mathbb{E}\left[Y \,|\, X\right]\bigr)^2 \;|\; X\Bigr]}{\mathbb{E}\left[ Y\,|\,X \right]} > 1.$$

This value is approximately equal to the residual deviance divided by the residual degrees of freedom, which in the linked question is $$\frac{1041.9}{8} \approx 130.2$$.

Printing a summary of the model shows that the estimated dispersion parameter is about $$135.4$$, fairly close.

## What does a quasipoisson GLM do?

This is nicely summarized, and compared to other approaches in an answer by AdamO:

[Quasipoisson models] maximize a "quasilikelihood" which is a Poisson likelihood up to a proportional constant. That proportional constant happens to be the dispersion.

This dispersion parameter, which provides the correction to the standard errors for overdispersion, is accounted for by the model. So in theory, there should be no upper bound for the amount of overdispersion for the quasipoisson approach to be 'valid'...

## As for the choice in the linked question...

The methods employed in the linked question involve estimating sandwich-based standard errors... so it actually doesn't matter whether quasipoisson, poisson, or even negative binomial is used:

The quasipoisson and Poisson models (always) give identical estimates, and we don't use their standard errors, because those are estimated separately. The negative binomial approach gives very slightly different estimates, but for all practical intents, they are equal.

(There are other approaches yet, like observation-level random effects, but the point remains.)

• For you to say that the mean/variance ratio is "irrelevant" contradicts many, many sources that explain or define overdispersion, including the Adam O answer you cite and the Ver Hoef/Boveng article I cite. Thus I'm struggling with this statement of yours. Commented Jul 14 at 11:14
• @rolando2, If the mean-variance relationship of the outcome, unconditional on the explanatory effects mattered, then we would have overdispersion for any Poisson regression with a non-zero slope. Commented Jul 14 at 11:31
• @rolando2 I'm afraid you are misreading the article. They also define things with $\mu$ as a variable, i.e. conditioned on $X$. This is analog to the idea of normal distribution in linear models stats.stackexchange.com/a/12266/341520 Commented Jul 14 at 12:01
• Counterexample: We obtain a nonzero slope with no overdispersion when we use x=c(0,0,0,0,4,4), y=c(0,0,1,1,2,2) mean(y); var(y) # 1.0 and 0.8 summary(testq <- glm(y~x,family=quasipoisson())) #Same 0.35 coefficient with poisson regression per se, but using quasipoisson prints out the dispersion parameter of 0.5 Commented Jul 14 at 12:14
• Thank you, but it's really not the only exception to your earlier statement, because one can use x=c(10,10,10,10,14,14,15); y=c(3,4,5,5,6,8,10), with mean and variance of y ~5.8, and one still gets the same pattern of results: no overdispersion and a nonzero slope. Commented Jul 14 at 14:51

If $$Y$$ is Poisson distributed, then $$\text{var}[Y] = E[Y]$$.

Imagine some derived variable $$X = kY$$ which scales the outcome by a fixed value $$k$$, the. We have the relationship $$\text{var}[X] = k E[X]$$.

This is one example of a process where any degree of over-dispersion is perfectly fine, and there is no limit to the ratio $$\frac{\text{var}[X]}{E[X]}$$ where we should rule out overdispersion and the use of a quasi-Poisson likelihood function.

Example, a pirate ship full of chests filled with coins exploded in front of the coast and the chests are spreading along the coast following a Poisson distribution, but we don't count the chests (say that they have rotten away) and instead we count the coins that came from the chest (say 500 coins in each chest with a 90% probability of finding a coin).

This could look like below. Here the observed variance is 270 times the mean.

set.seed(1)

# some variable on which the rate is dependent
# for example distance or depth
x = 5:15
n = length(x)

k_chests = rpois(n,x) # number of chests
k_coins = rep(NA,n)   # number of coins

# sulimulate the distribution of coins
for (i in 1:n) {
# coins is the sum of the coins from the chest, each coin has 90%
# probability to be found
k_coins[i] = sum(rbinom(k_chests[i],500, p = 0.9))
}

### dispersion parameter estimated at 270
mod = glm(k_coins ~ 1 + x, family= quasipoisson(link = "identity"))
summary(mod)

### wrong dispersion parameter of 1
### wrong estimate of p-values
mod = glm(k_coins ~ 1+x, family= poisson(link = "identity"))
summary(mod)

# plotting
plot(x,k_coins)
lines(x,predict(mod))


The question whether or not over-dispersion is good or not and whether a quasi-likelihood method is applicable, should not be made based on the degree of over-dispersion, but instead based on information about the process that generated the data.

In the referenced question, the over-dispersion seems to stem from the model itself being a bad estimate for the expectation value of a particular observation (there are several points that appear to be spread around a single line, and two points with a large difference). That inflates the variance and will partially explain the variance and related estimate of overdispersion. In that case the use of quasi-poison likelihood may not be so great, but not because of the large dispersion, and instead because of the irregular pattern in the residuals.

• Are you using a model-based definition of overdispersion or simply "variance (Y) > mean (Y)"? Commented Jul 14 at 16:57