I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated.

Okay, so the situation is this: I am to compare the prior distributions obtained using a binomial and a negative binomial, where (in both cases) there are $n$ trials and $m$ successes. I get the right answer for the binomial case, but not for the negative binomial.

Let's call the Jeffreys' prior $\pi_J(\theta)$. Then,

$$ \pi_J(\theta)\propto [I(\theta)]^{1/2}. $$

Under the regularity conditions (fulfilled as we are dealing with the exponential family),

$$ I(\theta)=-E\left(\frac{\partial^2 \log L(\theta|x)}{\partial \theta^2}\right) $$ where for the negative binomial $n$ is $x$ in the above expression (the total number of successes $m$ is fixed, $n$ is not). The distribution --I think-- is

$$ p(m|\theta)\propto\theta^m(1-\theta)^{n-m} $$ since $\theta$ is defined as the probability of success and $m$ is the number of successes. This is also the likelihood, since $m$ is a scalar and not a vector. Hence,

$$ L(\theta|n)\propto\theta^m(1-\theta)^{n-m}\\ \log L(\theta|n)=m\log\theta +(n-m)\log (1-\theta)\\ \frac{\partial\log L(\theta|n)}{\partial \theta}=\frac{m}{\theta}-\frac{n-m}{1-\theta}\\ \frac{\partial^2\log L(\theta|n)}{\partial \theta^2}=-\frac{m}{\theta^2}-\frac{n-m}{(1-\theta)^2} $$ so the Fisher information is

$$ I(\theta)=-E\left(\frac{\partial^2\log L(\theta|n)}{\partial \theta^2}\right)=\frac{m}{\theta^2}+\frac{E(n)-m}{(1-\theta)^2}=\frac{m}{\theta^2}+\frac{\frac{m\theta}{1-\theta}-m}{(1-\theta)^2}\\ =\frac{m(1-\theta)^2+\frac{m\theta^3}{(1-\theta)}-m\theta^2}{\theta^2(1-\theta)^2}=\frac{m(1-2\theta)+\frac{m\theta^3}{(1-\theta)}}{\theta^2(1-\theta)^2}\\ =\frac{m(1-2\theta)(1-\theta)+m\theta^3}{\theta^2(1-\theta)^3}=\frac{m(1-3\theta+2\theta^2+\theta^3)}{\theta^2(1-\theta)^3}\\ \propto\frac{1-3\theta+2\theta^2+\theta^3}{\theta^2(1-\theta)^3} $$

This, however, does not give me the correct answer. The correct answer is

$$ \pi_J(\theta)\propto \frac{1}{\theta(1-\theta)^{1/2}} $$ which means that the Information I get should be

$$ I(\theta)=\frac{1}{\theta^2(1-\theta)} $$ since the prior should be proportional to the square root of the information.

Can anyone find any mistakes? I wouldn't be surprised if I screwed something up with the set up of the distribution (successes vs failures with their respective probabilities, etc).

I used the expected value from Wikipedia and I know the correct answer from here (page 3).


1 Answer 1


The problem arises because the negative binomial distribution can be parametrized differently. As a consequence, the expectation differs for different parametrizations. The way you have specified the negative binomial distribution, the expectation of $n$ is $E(n) = m/\theta$ (e.g. see here on page 3). With that, the Fisher information simplifies to $$I(\theta) = m\left(\frac{1}{\theta^2(1-\theta)}\right)$$

Thus the Jeffreys' prior is $$ \pi_{J}(\theta) = |I(\theta)|^{1/2}\propto \theta^{-1}(1-\theta)^{-1/2} $$

as you already noted.

  • 1
    $\begingroup$ Terrific! That's very helpful and also an excellent reference as it goes through the very problem I was struggling with. Thank you! $\endgroup$
    – hejseb
    May 5, 2013 at 9:07
  • $\begingroup$ I've found a solution that uses another formulation, see here. Glad I could help. You're welcome. $\endgroup$ May 5, 2013 at 9:28
  • 1
    $\begingroup$ Archived version of the (now dead) link in the answer: web.archive.org/web/20180219125451/https://www.ida.liu.se/… $\endgroup$
    – JoshB
    Dec 19, 2022 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.