6
$\begingroup$

When $X$ is $X_1$,...$X_n$, how do you express the following negative binomial distribution in an exponential family form?

$$ f(k;r,p)\equiv\text{Pr}(X=k)=\binom{k+r-1}{k}(1-p)^{r}p^k~~~~~\text{for}~k=0,1,2,\ldots $$

$\endgroup$
  • 2
    $\begingroup$ Hey, I think you should be more specific as to what it is that you're doing and what your problem is. Your question, as it currently stands, probably won't get a very good or satisfying answer, and probably will get a number of downvotes. $\endgroup$ – Eric Peterson Jun 23 '13 at 14:27
  • $\begingroup$ Sorry, I am pretty new and don't know how to write an equation in LaTex. I just know how to use it for a very simple notation. I don't know how specific the question needs to be. Any suggestion? $\endgroup$ – Python_R Jun 23 '13 at 15:11
  • 1
    $\begingroup$ It's not clear to me what your question is, @Python_R. Can you say more about what you want to know? $\endgroup$ – gung - Reinstate Monica Jun 23 '13 at 15:34
16
$\begingroup$

Warning: The negative binomial distribution has several alternative formulations for which the formulas below change.

A distribution $f(x;\theta)$ belongs to an exponential family if it can be represented in the form:

$$ f(x;\theta)=h(x)\exp\left[\eta(\theta)T(x)-A(\theta)\right] $$ The value $\eta$ is called the canonical (natural) parameter of the family, $T(x)$ is a sufficient statistic for $p$, $A(\theta)$ is called the log-partition function (it's a normalization factor and sometimes called log-normalizer or cumulant generating function), and $h(x)$ is an arbitrary function called base measure or carrier measure which is 1 in many cases (e.g. exponential distribution, gamma distribution, Bernoulli distribution, ...). If the negative binomial distribution with known parameter $r$ (if $r$ is unknown, the negative binomial family is not an exponential family) has the following distribution:

$$ f(k;r,p)=\binom{k+r-1}{k}(1-p)^{r}p^k~~~~~\text{for}~k=0,1,2,\ldots $$ Then it can be rewritten in exponential form as:

$$ \begin{align} f(k;r,p) &=\binom{k+r-1}{k}\exp\left[\ln(p^{k}(1-p)^{r})\right] \\ &=\binom{k+r-1}{k}\exp\left[k\ln(p) + r\ln(1-p)\right] \\ \end{align} $$ So the parameter $\theta$ of the distribution is $p$ (i.e. $\theta=p$) and the natural parameter is $\eta=\ln(p)$, the sufficient statistic for $p$ is $T(k)=k$ (i.e. $T(k)=\sum X_{i}$, the sample sum), the log-partition function is $A(\eta)=-r\ln(1-p)$ and $h(k)=\binom{k+r-1}{k}$. See for example the Wikipedia page for a nice overview of the theory and many common distributions. Nice references can also be found here and here.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ +1. Also, $h(x)$ is called the carrier measure because in the measure-theoretic formulation of exponential families (e.g. Shao's book), it is an actual measure. $\endgroup$ – Neil G Jun 23 '13 at 23:07
  • $\begingroup$ @NeilG Thanks Neil, I didn't know that. Do you mean Jun Shao's book Mathematical statistics? If yes, could you point out where in the book? Thanks very much. $\endgroup$ – COOLSerdash Jun 24 '13 at 6:16
  • $\begingroup$ @COOLSerdash: Yes, that's the one. I don't have the book with me, but I remember seeing the measure theoretic formulation there. Just doing a quick Google search, you can find a reference to "carrier measure" in "The Geometry of Exponential Families" by Bradley Efron $\endgroup$ – Neil G Jun 24 '13 at 8:18
  • $\begingroup$ The carrier measure can usually be made into a standard carrier measure (by rescaling the sample space) for a continuous distribution, but is rarely equal to one for a discrete distribution. $\endgroup$ – Neil G Jun 24 '13 at 8:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.