Meaning of "Overdispersion" in Statistics

Question

I am trying to understand what "overdispersion" means in statistics.

Based on the Wikipedia page, "overdispersion" is defined as follows : "In statistics, overdispersion is the presence of greater variability (statistical dispersion) in a data set than would be expected based on a given statistical model."

However, I have heard other interpretations of "overdispersion" which suggest that "overdispersion refers to situations where the variance within the data is a function of the mean" - in other words, there is a non-constant relationship between the mean and variance within the data.

My Question: Can someone please tell how to mathematically measure and define "overdispersion"? For instance, I have heard that the Normal Distribution and the Poisson Distribution can be defined as "Dispersion Models". I have also heard that many models can be considered as "Dispersion Models" so long as a "Dispersion Parameter" can be inserted into the model. Using these definitions - is the Normal Distribution an example of "Overdispersion"? For example, there is a lot more variation in a Normal Distribution around the peak - and relatively less variation in a Normal Distribution around the tails. Is all this correct?

References:

• https://en.wikipedia.org/wiki/Overdispersion

Answer 1

In a Poisson$(\lambda)$ distribution:

$$ \mu=\lambda\\ \sigma^2 =\lambda\\ \implies\\ \mu=\sigma^2 $$

Consequently, when we believe we have a Poisson distribution, we expect the samples drawn from it to obey $\bar x \approx s^2$, since $\mu=\sigma^2$ in the suspected distribution.

If we have a gross violation where $s^2>>\bar x$, then we would not find it believable that $\mu=\sigma^2$, and we describe the data as overdispersed. That is, the dispersion is higher than we expected it to be.

Answer 2

For many one-parameter probability distributions, the variance in the distribution is a function of the mean. When you fit data to a statistical model using these distributions, the estimator will tend to give you a reasonable estimate of the mean, but the estimated variance will just be a function of that, so it will not generally fit to the data very well. This happens with certain one-parameter probability distributions, most notably the Poisson distribution. In this case, it is common for the data to be more variable than the estimated variance coming out of your model, in which case we say that there is a problem of "overdispersion".

Both of the descriptions you have given for this are correct. Overdispersion is indeed the presence of greater variability in the data than predicted by the model. This generally occurs because the variance in the distribution used in the model is a function of the mean, so the estimation procedure can't estimate them both well (and mean estimation is generally more important than variance estimation when fitting data to a model).

Roughly speaking, if you have $k$ parmaters in a statistical distribution, and you fit it to data, it will allow you to accurately estimate $k$ moments of the distribution (often the first $k$ moments, but not always$^\dagger$). So, for example, some one-parameter distributions allow you to accurately estimate the mean but not the variance, some two-parameter distributions allow you to accurately estimate the mean and variance but not the skewness, some three-parameter distributions allow you to accurately estimate the mean, variance and skewness, but not the kurtosis, and so on.

If you want to avoid overdispersion in your modelling, you should use statistical models that use an underlying two-parameter distribution that can fit the mean and variance (e.g., use a negative binomial model instead of a Poisson model). The same basic principle also applies if you want to accurately fit higher-order moments --- e.g., if you want to accurately fit skewness you might generalise to a three-parameter distribution, and so on.

---

$^\dagger$ For example, the Student's T-distribution has a single parameter that affects the variance and kurtosis but not the mean or skewness.

Answer 3

The only place I run into over-dispersion issues is when e.g. fitting a GLM model based on Poisson count data (Poisson regression).

As you know, for Poisson, the variance is equal to the mean, so

\begin{equation} \mathrm{Var}(Y_i) = \mathrm{E}(Y_i). \end{equation}

But commonly the variance exceeds the mean, so an attempt is made to recover the relationship above by introducing an over-dispersion parameter, $\phi$ in the functional form

\begin{equation} \mathrm{Var}(Y_i) = \phi\mathrm{E}(Y_i), \end{equation}

which is fitted along with the parameters during maximization of the log-likelihood. For Poisson, $\phi=1$, and if you allow $\phi>1$, you no longer have a distribution from the exponential family. The end result of implementing $\phi$ is that $V(\beta)$ is purposely inflated, leading to an over-estimation of standard errors -- essentially an under-statement of significance of the parameters. By not taking $\phi$ into account, it leads to underestimated standard errors, or over-statement about significance.

Another work-around is to use quantile regression, which is hinged to ranks (non-parametrics).

Answer 4

Over-dispersion can occur with one-parameter distributions, where mean and variance are tied together (Poisson, Binomial, Exponential). In real data, variance is usually much greater than would be allowed. Over-dispersion creates over-confidence (e.g. too narrow CIs), but usually does not introduce biases. In practical modelling, this problem can be resolved in one of three ways:

1. quasi-likelihood or generalized equation estimation
2. two-parameter distributions, such as negative-binomial or beta-binomial
3. observation-level random effects

I am discussing the issue and solutions 2 & 3 in my book.