3
$\begingroup$

I am studying the negative binomial distribution and it seems it models overdispersion in a poisson process using a gamma distribution (a mixture or something, but I may be wrong).

In parallel, I am studying about Bayesian inference and it caught my attention that when modelling a Poisson process the prior is drawn from a gamma distribution. From my beginner understanding Bayesian statistics are particularly useful to study cases with overdispersion (few observations). Hence, I was wondering whether there is a relationship between the neg-bin distribution and the Bayesian modelling of a Poisson process. I am just trying to get a deeper understanding. Thanks!

$\endgroup$
4
$\begingroup$

(Note that in the following I am quite careful about when to use capital $\Lambda$ and when to use lower-case $\lambda$, and when to use capital $X$ and when to use lower-case $x$.)

Suppose the distribution of $\Lambda$ is the Gamma distribution: $$ \frac 1 {\Gamma(\alpha)} \left( \frac \lambda \beta \right)^{\alpha-1} e^{-\lambda/\beta} \left(\frac{d\lambda}\beta\right). $$ And suppose the conditional distribution of $X$ given $\Lambda$ is the Poisson distribution $$ x \mapsto \frac{e^{-\Lambda} \Lambda^x }{x!} \text{ for } x = 0,1,2,3,\ldots $$ Then you have \begin{align} & \Pr(X=x) \\[10pt] = {} & \operatorname{E}(\Pr(X=x \mid \Lambda)) = \operatorname{E}\left( \frac{e^{-\Lambda} \Lambda^x}{x!} \right) \\[10pt] = {} & \int_0^\infty \left( \frac{e^{-\lambda}\lambda^x}{x!} \right) \frac 1 {\Gamma(\alpha)} \left( \frac \lambda \beta \right)^{\alpha-1} e^{-\lambda/\beta} \left(\frac{d\lambda}\beta\right) \\[10pt] = {} & \frac 1 {x!\Gamma(\alpha)\beta^\alpha} \int_0^\infty \lambda^{x+\alpha-1} e^{-\lambda(1\,+\,1/\beta)} \, d\lambda \\ & \qquad \qquad \text{(We pulled out the factors that do not depend on $\lambda$).}\\[10pt] = {} & \frac 1 {x!\Gamma(\alpha)\beta^\alpha} \cdot \left(\frac \beta {\beta+1} \right)^{x+\alpha} \int_0^\infty \left( \lambda \left( 1 + \frac 1 \beta \right) \right)^{x+\alpha-1} e^{-\lambda(1+1/\beta)} \left( d\lambda \left( 1 + \frac 1 \beta \right) \right) \\[10pt] = {} & \frac 1 {x!\Gamma(\alpha)\beta^\alpha} \cdot \left(\frac \beta {\beta+1} \right)^{x+\alpha} \int_0^\infty u^{x+\alpha-1} e^{-u} \, du \\[10pt] = {} & \frac {\Gamma(x+\alpha)} {x!\Gamma(\alpha)\beta^\alpha} \cdot \left(\frac \beta {\beta+1} \right)^{x+\alpha} \\[10pt] = {} & \frac {\Gamma(x+\alpha)} {x!\Gamma(\alpha)} \cdot \left( \frac 1 {\beta+1} \right)^\alpha \left(\frac \beta {\beta+1} \right)^x \end{align} and this is a negative binomial probability mass function with $p$ and $q$ equal to $\dfrac 1 {\beta+1}$ and $\dfrac\beta{\beta+1}$, not necessarily respectively (depending on which convention you follow). This is the probability that the number of failures before the $\alpha$th success is $x$, when the probability of success on each independent trial is $1/(\beta+1).$

So a Gamma mixture of Poisson distributions is a negative binomial distribution, and that is the connection between Poisson and negative binomial.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks to both, Student T and Michael Hardy! So, could we say that the negative binomial is both a frequentist approach and a Bayesian approach? Could we say that in this case frequentist and Bayesian approaches match each other? $\endgroup$ – Federico Feb 21 '17 at 11:22
  • $\begingroup$ @Federico : A mere probability distribution is not Bayesian or frequentist, even if it is used more often in one of those approaches than the other. Bayesianism is about assigning probabilities to everything that is uncertain, regardless of whether it is random. The probability that there was life on Mars a billion years ago may be $0.6$ regardless of whether it is true in $60\%$ of all cases. $\endgroup$ – Michael Hardy Feb 21 '17 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.