# Neg Binomial and the Jeffreys' Prior

I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated.

Okay, so the situation is this: I am to compare the prior distributions obtained using a binomial and a negative binomial, where (in both cases) there are $n$ trials and $m$ successes. I get the right answer for the binomial case, but not for the negative binomial.

Let's call the Jeffreys' prior $\pi_J(\theta)$. Then,

$$\pi_J(\theta)\propto [I(\theta)]^{1/2}.$$

Under the regularity conditions (fulfilled as we are dealing with the exponential family),

$$I(\theta)=-E\left(\frac{\partial^2 \log L(\theta|x)}{\partial \theta^2}\right)$$ where for the negative binomial $n$ is $x$ in the above expression (the total number of successes $m$ is fixed, $n$ is not). The distribution --I think-- is

$$p(m|\theta)\propto\theta^m(1-\theta)^{n-m}$$ since $\theta$ is defined as the probability of success and $m$ is the number of successes. This is also the likelihood, since $m$ is a scalar and not a vector. Hence,

$$L(\theta|n)\propto\theta^m(1-\theta)^{n-m}\\ \log L(\theta|n)=m\log\theta +(n-m)\log (1-\theta)\\ \frac{\partial\log L(\theta|n)}{\partial \theta}=\frac{m}{\theta}-\frac{n-m}{1-\theta}\\ \frac{\partial^2\log L(\theta|n)}{\partial \theta^2}=-\frac{m}{\theta^2}-\frac{n-m}{(1-\theta)^2}$$ so the Fisher information is

$$I(\theta)=-E\left(\frac{\partial^2\log L(\theta|n)}{\partial \theta^2}\right)=\frac{m}{\theta^2}+\frac{E(n)-m}{(1-\theta)^2}=\frac{m}{\theta^2}+\frac{\frac{m\theta}{1-\theta}-m}{(1-\theta)^2}\\ =\frac{m(1-\theta)^2+\frac{m\theta^3}{(1-\theta)}-m\theta^2}{\theta^2(1-\theta)^2}=\frac{m(1-2\theta)+\frac{m\theta^3}{(1-\theta)}}{\theta^2(1-\theta)^2}\\ =\frac{m(1-2\theta)(1-\theta)+m\theta^3}{\theta^2(1-\theta)^3}=\frac{m(1-3\theta+2\theta^2+\theta^3)}{\theta^2(1-\theta)^3}\\ \propto\frac{1-3\theta+2\theta^2+\theta^3}{\theta^2(1-\theta)^3}$$

This, however, does not give me the correct answer. The correct answer is

$$\pi_J(\theta)\propto \frac{1}{\theta(1-\theta)^{1/2}}$$ which means that the Information I get should be

$$I(\theta)=\frac{1}{\theta^2(1-\theta)}$$ since the prior should be proportional to the square root of the information.

Can anyone find any mistakes? I wouldn't be surprised if I screwed something up with the set up of the distribution (successes vs failures with their respective probabilities, etc).

I used the expected value from Wikipedia and I know the correct answer from here (page 3).

The problem arises because the negative binomial distribution can be formulated differently. As a consequence, the expectation differs for different formulations. The way you have specified the negative binomial distribution, the expectation of $n$ is $E(n) = m/\theta$ (e.g. see here on page 3). With that, the Fisher information simplifies to $$I(\theta) = m\left(\frac{1}{\theta^2(1-\theta)}\right)$$
Thus the Jeffreys' prior is $$\pi_{J}(\theta) = |I(\theta)|^{1/2}\propto \theta^{-1}(1-\theta)^{-1/2}$$