I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated.

Okay, so the situation is this: I am to compare the prior distributions obtained using a binomial and a negative binomial, where (in both cases) there are $n$ trials and $m$ successes. I get the right answer for the binomial case, but not for the negative binomial.

Let's call the Jeffreys' prior $\pi_J(\theta)$. Then,

$$ \pi_J(\theta)\propto [I(\theta)]^{1/2}. $$

Under the regularity conditions (fulfilled as we are dealing with the exponential family),

$$ I(\theta)=-E\left(\frac{\partial^2 \log L(\theta|x)}{\partial \theta^2}\right) $$ where for the negative binomial $n$ is $x$ in the above expression (the total number of successes $m$ is fixed, $n$ is not). The distribution --I think-- is

$$ p(m|\theta)\propto\theta^m(1-\theta)^{n-m} $$ since $\theta$ is defined as the probability of success and $m$ is the number of successes. This is also the likelihood, since $m$ is a scalar and not a vector. Hence,

$$ L(\theta|n)\propto\theta^m(1-\theta)^{n-m}\\ \log L(\theta|n)=m\log\theta +(n-m)\log (1-\theta)\\ \frac{\partial\log L(\theta|n)}{\partial \theta}=\frac{m}{\theta}-\frac{n-m}{1-\theta}\\ \frac{\partial^2\log L(\theta|n)}{\partial \theta^2}=-\frac{m}{\theta^2}-\frac{n-m}{(1-\theta)^2} $$ so the Fisher information is

$$ I(\theta)=-E\left(\frac{\partial^2\log L(\theta|n)}{\partial \theta^2}\right)=\frac{m}{\theta^2}+\frac{E(n)-m}{(1-\theta)^2}=\frac{m}{\theta^2}+\frac{\frac{m\theta}{1-\theta}-m}{(1-\theta)^2}\\ =\frac{m(1-\theta)^2+\frac{m\theta^3}{(1-\theta)}-m\theta^2}{\theta^2(1-\theta)^2}=\frac{m(1-2\theta)+\frac{m\theta^3}{(1-\theta)}}{\theta^2(1-\theta)^2}\\ =\frac{m(1-2\theta)(1-\theta)+m\theta^3}{\theta^2(1-\theta)^3}=\frac{m(1-3\theta+2\theta^2+\theta^3)}{\theta^2(1-\theta)^3}\\ \propto\frac{1-3\theta+2\theta^2+\theta^3}{\theta^2(1-\theta)^3} $$

This, however, does not give me the correct answer. The correct answer is

$$ \pi_J(\theta)\propto \frac{1}{\theta(1-\theta)^{1/2}} $$ which means that the Information I get should be

$$ I(\theta)=\frac{1}{\theta^2(1-\theta)} $$ since the prior should be proportional to the square root of the information.

Can anyone find any mistakes? I wouldn't be surprised if I screwed something up with the set up of the distribution (successes vs failures with their respective probabilities, etc).

I used the expected value from Wikipedia and I know the correct answer from here (page 3).


The problem arises because the negative binomial distribution can be formulated differently. As a consequence, the expectation differs for different formulations. The way you have specified the negative binomial distribution, the expectation of $n$ is $E(n) = m/\theta$ (e.g. see here on page 3). With that, the Fisher information simplifies to $$I(\theta) = m\left(\frac{1}{\theta^2(1-\theta)}\right)$$

Thus the Jeffreys' prior is $$ \pi_{J}(\theta) = |I(\theta)|^{1/2}\propto \theta^{-1}(1-\theta)^{-1/2} $$

as you already noted.

  • 1
    $\begingroup$ Terrific! That's very helpful and also an excellent reference as it goes through the very problem I was struggling with. Thank you! $\endgroup$ – hejseb May 5 '13 at 9:07
  • $\begingroup$ I've found a solution that uses another formulation, see here. Glad I could help. You're welcome. $\endgroup$ – COOLSerdash May 5 '13 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.