10
$\begingroup$

I am trying to understand what "overdispersion" means in statistics.

Based on the Wikipedia page, "overdispersion" is defined as follows : "In statistics, overdispersion is the presence of greater variability (statistical dispersion) in a data set than would be expected based on a given statistical model."

However, I have heard other interpretations of "overdispersion" which suggest that "overdispersion refers to situations where the variance within the data is a function of the mean" - in other words, there is a non-constant relationship between the mean and variance within the data.

My Question: Can someone please tell how to mathematically measure and define "overdispersion"? For instance, I have heard that the Normal Distribution and the Poisson Distribution can be defined as "Dispersion Models". I have also heard that many models can be considered as "Dispersion Models" so long as a "Dispersion Parameter" can be inserted into the model. Using these definitions - is the Normal Distribution an example of "Overdispersion"? For example, there is a lot more variation in a Normal Distribution around the peak - and relatively less variation in a Normal Distribution around the tails. Is all this correct?

Thanks!

References:

$\endgroup$
2
  • 1
    $\begingroup$ With something that might be expected t be Poisson, overdispersion is specifically for the situation where the variance is greater than mean (by comparison with the Poisson, where in the population the variance is equal the mean), not just varying with it. Consider the binomial, for example, which have variance that is a function of the mean but which is always less than the mean (by a factor of $1-p$). That would not be called overdispersed relative to the Poisson -- quite the opposite. $\endgroup$
    – Glen_b
    Dec 4 '21 at 6:13
  • 1
    $\begingroup$ More generally, anything where the variance is larger than some model would suggest might be called overdispersed, by extension from the Poisson case (which is where the term originates). In a GLM context you can estimate dispersion, but it will only be called overdispersed for cases where the dispersion parameter is already specified (binomial and Poisson, most typically) $\endgroup$
    – Glen_b
    Dec 4 '21 at 6:19
17
$\begingroup$

In a Poisson$(\lambda)$ distribution:

$$ \mu=\lambda\\ \sigma^2 =\lambda\\ \implies\\ \mu=\sigma^2 $$

Consequently, when we believe we have a Poisson distribution, we expect the samples drawn from it to obey $\bar x \approx s^2$, since $\mu=\sigma^2$ in the suspected distribution.

If we have a gross violation where $s^2>>\bar x$, then we would not find it believable that $\mu=\sigma^2$, and we describe the data as overdispersed. That is, the dispersion is higher than we expected it to be.

$\endgroup$
8
$\begingroup$

For many one-parameter probability distributions, the variance in the distribution is a function of the mean. When you fit data to a statistical model using these distributions, the estimator will tend to give you a reasonable estimate of the mean, but the estimated variance will just be a function of that, so it will not generally fit to the data very well. This happens with certain one-parameter probability distributions, most notably the Poisson distribution. In this case, it is common for the data to be more variable than the estimated variance coming out of your model, in which case we say that there is a problem of "overdispersion".

Both of the descriptions you have given for this are correct. Overdispersion is indeed the presence of greater variability in the data than predicted by the model. This generally occurs because the variance in the distribution used in the model is a function of the mean, so the estimation procedure can't estimate them both well (and mean estimation is generally more important than variance estimation when fitting data to a model).

Roughly speaking, if you have $k$ parmaters in a statistical distribution, and you fit it to data, it will allow you to accurately estimate $k$ moments of the distribution (often the first $k$ moments, but not always$^\dagger$). So, for example, some one-parameter distributions allow you to accurately estimate the mean but not the variance, some two-parameter distributions allow you to accurately estimate the mean and variance but not the skewness, some three-parameter distributions allow you to accurately estimate the mean, variance and skewness, but not the kurtosis, and so on.

If you want to avoid overdispersion in your modelling, you should use statistical models that use an underlying two-parameter distribution that can fit the mean and variance (e.g., use a negative binomial model instead of a Poisson model). The same basic principle also applies if you want to accurately fit higher-order moments --- e.g., if you want to accurately fit skewness you might generalise to a three-parameter distribution, and so on.


$^\dagger$ For example, the Student's T-distribution has a single parameter that affects the variance and kurtosis but not the mean or skewness.

$\endgroup$
7
$\begingroup$

The only place I run into over-dispersion issues is when e.g. fitting a GLM model based on Poisson count data (Poisson regression).

As you know, for Poisson, the variance is equal to the mean, so

\begin{equation} \mathrm{Var}(Y_i) = \mathrm{E}(Y_i). \end{equation}

But commonly the variance exceeds the mean, so an attempt is made to recover the relationship above by introducing an over-dispersion parameter, $\phi$ in the functional form

\begin{equation} \mathrm{Var}(Y_i) = \phi\mathrm{E}(Y_i), \end{equation}

which is fitted along with the parameters during maximization of the log-likelihood. For Poisson, $\phi=1$, and if you allow $\phi>1$, you no longer have a distribution from the exponential family. The end result of implementing $\phi$ is that $V(\beta)$ is purposely inflated, leading to an over-estimation of standard errors -- essentially an under-statement of significance of the parameters. By not taking $\phi$ into account, it leads to underestimated standard errors, or over-statement about significance.

Another work-around is to use quantile regression, which is hinged to ranks (non-parametrics).

$\endgroup$
3
  • 1
    $\begingroup$ Over- (and under-) dispersion is also reasonably common in (quasi-)binomial cases; the binomial distribution suggests that the variance should be $np(1-p)$ but in practice it is significantly more or less than that. $\endgroup$
    – JDL
    Dec 3 '21 at 13:55
  • 2
    $\begingroup$ "Poisson is the only distribution for which the mean is equal to the variance" seems to be a bit sweeping (consider distributions with location and scale parameters) $\endgroup$
    – Henry
    Dec 3 '21 at 15:42
  • $\begingroup$ @Henry - changed to "As you know, under the Poisson distribution, the mean is equal to the variance, so..." $\endgroup$ Dec 4 '21 at 1:13
0
$\begingroup$

Over-dispersion can occur with one-parameter distributions, where mean and variance are tied together (Poisson, Binomial, Exponential). In real data, variance is usually much greater than would be allowed. Over-dispersion creates over-confidence (e.g. too narrow CIs), but usually does not introduce biases. In practical modelling, this problem can be resolved in one of three ways:

  1. quasi-likelihood or generalized equation estimation
  2. two-parameter distributions, such as negative-binomial or beta-binomial
  3. observation-level random effects

I am discussing the issue and solutions 2 & 3 in my book.

$\endgroup$
2
  • 2
    $\begingroup$ or quasi-likelihood modeling ... $\endgroup$
    – Ben Bolker
    Dec 4 '21 at 1:34
  • 5
    $\begingroup$ Your answers are getting flagged because they both have links to sites you control. Even though most people wouldn't consider this to be spam or even bad (why not link to your own material?), it's best to acknowledge this relationship in your answers. $\endgroup$
    – whuber
    Dec 4 '21 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.