5
$\begingroup$

can anyone show me how to find the Fisher information matrix for negative binomial distribution, if I parameterize the Negative Binomial distribution using parameters mean and size.

$\endgroup$
3
  • $\begingroup$ Here is a link to a page where they derive the Fisher information for the negative binomial: www2.isye.gatech.edu/~brani/isyebayes/bank/Quiz71.pdf Hope it helps, $\endgroup$
    – freddyfree
    May 14, 2014 at 19:30
  • 2
    $\begingroup$ It helps. But it didn't give the fisher information MATRIX. only fisher information for $\theta$ $\endgroup$
    – Q_Li
    May 14, 2014 at 20:09
  • $\begingroup$ OP wrote: Negative Binomial distribution using parameters mean and size ...\\\\\\\\\\\\\\\\\\ -> Please specify the functional form for the pmf that you are working with. $\endgroup$
    – wolfies
    May 15, 2014 at 9:20

1 Answer 1

3
$\begingroup$

The negative binomial distribition parametrized by mean and size can be given by $$ \DeclareMathOperator{\P}{\mathbb{P}} \P (X=k) = \binom{k+m-1}{k}\left( \frac{m}{m+\mu} \right)^m \left( \frac{\mu}{m+\mu} \right)^k $$ for the outcome $k$ a nonnegative integer, and $\mu>0$ the mean, $m>0$ the size. I will do the calculations by maple.

The Fisher information matrix (of size $2\times 2$) has components $I_{\mu\mu}, I_{\mu m} \text{ and } I_{m m}$ given by $$ \DeclareMathOperator{\E}{\mathbb{E}} I_{ij}=-\E\left\{ \frac{\partial^2}{\partial \theta_i \partial\theta_j}\log f(X;\theta)|\theta \right\} $$ where here $\theta=(\mu, m)$. Then we (maple code at the end of post) find $$ I_{\mu\mu}=\frac{m}{(m+\mu)\mu} $$ The diagonal term is simplest, it reduces to zero! That is the beauty of the mean parametrization $$ I_{\mu m}=0 $$ showing that $\mu$ and $m$ are orthogonal parameters.

For the last term, the result will involve a trigamma function written $\Psi(1,\cdot)$ (second derivative of log of gamma function) and result will be a somewhat complex infinite series, which must be evaluated numerically: $$ I_{mm}=\sum_{k=0}^\infty \binom{k+m-1}{k}\left\{ -m^{m-1}\mu^k (m+\mu)^{-m-2-k} \left( m(m+\mu)^2 \Psi(1,k+m) -m(m+\mu)^2 \Psi(1,m) +mk+\mu^2 \right) \right\} $$ A concise form can be derived either by simplifying the expression that Maple given above:

\begin{align*} I_{mm} =& -\sum_{k=0}^\infty\binom{k+m-1}{k}\left(\frac{m}{m+\mu}\right)^m \left(\frac{\mu}{m+\mu}\right)^k\{\frac{1}{(m+\mu)^2m}\left(m(m+\mu)^2\Psi(1,k+m)-m(m+\mu)^2\Psi(1,m)+m k+\mu^2\right)\}\\ =& -\mathbb{E}\left(\frac{1}{(m+\mu)^2m}\left(m(m+\mu)^2\Psi(1,X+m)-m(m+\mu)^2\Psi(1,m)+m X+\mu^2\right)\right)\\ =& -\mathbb{E}\left(\frac{1}{(m+\mu)^2m}\{m(m+\mu)^2(\Psi(1,X+m) - \Psi(1,m))+m X +\mu^2\}\right)\\ =& -\mathbb{E}\left(\Psi(1,X+m) - \Psi(1,m)\right) - \frac{\mu}{m(m+\mu)} \end{align*} where $X$ follows negative binomial distribution with mean $\mu$ and size $m$.

Or by definition of Fisher information \begin{align*} I_{mm} =& - \mathbb{E}\frac{\partial^2}{\partial m^2}\ln \mathbb{P}(X;\mu,m) \\ =& - \mathbb{E}\frac{\partial}{\partial m} \{\Psi(X+ m) - \Psi( m) + \ln\frac{ m}{ m+\mu} + \frac{\mu -X }{ m+ \mu}\}\\ =& - \mathbb{E} \{\frac{\partial}{\partial m}(\Psi(X+ m) - \Psi( m)) + + \frac{1}{ m}-\frac{1}{ m+\mu}-\frac{\mu - X}{( m+\mu)^2}\}\\ =& -\mathbb{E}\frac{\partial}{\partial m}\left(\Psi(X+ m) - \Psi( m)\right) -\frac{\mu}{ m( m+\mu)} \\ =& -\mathbb{E}\left(\Psi(1,X+ m) - \Psi(1, m)\right) -\frac{\mu}{ m( m+\mu)} \end{align*} where $\Psi(\cdot)$ is the digamma function (first derivative of log of gamma function).

Below some maple code (and output):

f :=  binomial(k+m-1,k)*(m/(m+mu))^m * (mu/(m+mu))^k
                                            m         k
                                    /  m   \  /  mu  \ 
        f := binomial(k + m - 1, k) |------|  |------| 
                                    \m + mu/  \m + mu/ 
lf := ln( binomial(k+m-1,k) ) + m*ln( m/(m+mu) ) + k* ln( mu/(m+mu) ) assuming m>0,mu>0;
                                        /  m   \       /  mu  \
 lf := ln(binomial(k + m - 1, k)) + m ln|------| + k ln|------|
                                        \m + mu/       \m + mu/

simplify( -sum(f*diff(lf,mu,mu),k=0..infinity ) ) assuming m>0,mu>0;
                               m     
                          -----------
                          (m + mu) mu

simplify( -sum(f*diff(lf,mu,m),k=0..infinity ) ) assuming m>0,mu>0;
                               0

simplify( -sum(f*diff(lf,m,m),k=0..infinity ) ) assuming m>0,mu>0;
 infinity                                                         
  -----                                                           
   \                                                              
    )     (m - 1)         (-m - 2 - k)   k                        
-  /     m        (m + mu)             mu  binomial(k + m - 1, k) 
  -----                                                           
  k = 0                                                           

  /          2                           2                     2\
  \m (m + mu)  Psi(1, k + m) - m (m + mu)  Psi(1, m) + m k + mu /
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.