Negative binomial distribution mean and variance

Question

The equation below indicates expected value of negative binomial distribution. I need a derivation for this formula. I have searched a lot but can't find any solution. Thanks for helping :)

$ E(X)=\sum _{x=r}^{}x\times \left(\begin{array}{c}x-1\\ r-1\end{array}\right)\times {p}^{r}\times (1-p{)}^{x-r}=\frac{r}{p} $

I have tried: \begin{align} E(X)&=\sum _{x=r}^{}x\times \left(\begin{array}{c}x-1\\ r-1\end{array}\right)\times {p}^{r}\times (1-p{)}^{x-r}\\&=\sum _{x=r}^{}x\times \frac{(x-1)!}{(r-1)!\times ((x-1-(r-1))!}\times {p}^{r}\times (1-p{)}^{x-r}\\ &=\sum _{x=r}^{}\frac{x!}{(r-1)!\times ((x-r)!}\times {p}^{r}\times (1-p{)}^{x-r}\\ &\Rightarrow \phantom{\rule{0ex}{0ex}}\sum _{x=r}^{}r\times \frac{x!}{r!\times (x-r)!}\times {p}^{r}\times (1-p{)}^{x-r}\\ &=\frac{r}{p}\times \sum _{x=r}^{}\frac{x!}{r!\times (x-r)!}\times {p}^{r+1}\times (1-p{)}^{x-r} \end{align}

If the power of p in the last equation were not r + 1, I can implement Newton Binomial. So It will be true. But I am stuck here.

Answer 1

I would like to complement whuber's answer by a bit longer but purely arithmetical solution:

\begin{align} E(X)&=\sum _{x=r}^{\infty}x \frac{(x-1)!}{(r-1)! (x-r)!}\times {p}^{r} (1-p{)}^{x-r}\\ &=\sum _{x=r}^{\infty}(x - r + r) \frac{(x-1)!}{(r-1)! (x-r)!}\times {p}^{r} (1-p{)}^{x-r}\\ &=\sum _{x=r}^{\infty}(x - r) \frac{(x-1)!}{(r-1)! (x-r)!}\times {p}^{r}(1-p)^{x-r} + r \sum _{x=r}^{\infty}\frac{(x-1)!}{(r-1)! (x-r)!}\times {p}^{r} (1-p{)}^{x-r}\\ &=\sum _{x=r + 1}^{\infty}\frac{(x-1)!}{(r-1)! (x - r - 1)!}\times {p}^{r}(1-p)^{x-r} + r \sum _{x=r}^{\infty}\left(\begin{array}{c}x-1\\ r-1\end{array}\right)\times {p}^{r} (1-p)^{x-r}\\ &=\sum _{x=r + 1}^{\infty}\frac{r(1 - p)}{p}\frac{(x-1)!}{r! (x - r - 1)!}\times {p}^{r+1}(1-p)^{x-r-1} + r\\ &=\frac{r(1 - p)}{p}\sum _{x=r + 1}^{\infty}\left(\begin{array}{c}x-1\\ r\end{array}\right)\times {p}^{r+1}(1-p)^{x-r-1} + r\\ &=\frac{r(1 - p)}{p} + r\\ & = \frac{r}{p}. \end{align}

Here we twice used the fact that the sum of all of the probabilies of a discrete random variable is equal to one:

$$\sum _{x=r}^{\infty}\left(\begin{array}{c}x-1\\ r-1\end{array}\right)\times {p}^{r} (1-p)^{x-r} = \sum _{x=r}^{\infty} \mathbb{P}(X = x) = 1,$$

where $X$ is a negative binomial with parameters $r$ and $p$, and similarly

$$\sum _{x=r + 1}^{\infty}\left(\begin{array}{c}x-1\\ r\end{array}\right)\times {p}^{r+1}(1-p)^{x-r-1} = \sum _{x=r+1}^{\infty} \mathbb{P}(X^\prime = x) = 1,$$

where $X^\prime$ is a negative binomial with parameters $r + 1$ and $p$.

Note: In all of the calculations above, I was using the notation given in the question. It is worth mentioning that there are at least two different ways to define a negative binomial distribution: either $X$ counts the number of failures, given $r$ successes (this is the most common definition), or $X$ counts the number of overall trials, given $r$ successes. The question implicitly assumes the second definition, whereas whuber's answer provides solution for the first one. Hence, the answers are different but are consistent with each other. In the first case, $E(X) = \frac{r(1-p)}{p}$ represents the average number of failures before $r$ successes, whereas in the second case $E(X) = \frac{r}{p}$ stands for the average number of trials with $r$ successes. Clearly, $$\frac{r(1-p)}{p} + r = \frac{r}{p}.$$

Answer 2

Consider the Negative Binomial distribution with parameters $r\gt 0$ and $0\lt p\lt 1.$ According to one definition, it has positive probabilities for all natural numbers $k\ge 0$ given by

$$\Pr(k\mid r, p) = \binom{-r}{k}(-1)^k (1-p)^r\,p^k.$$

Newton's Binomial Theorem states that when $|q|\lt 1$ and $x$ is any number,

$$(1+q)^x = \sum_{k=0}^\infty \binom{x}{k} q^k.$$

Because this sum converges absolutely it can be differentiated term by term, giving

$$qx(1+q)^{x-1} = q\frac{d}{dq}(1+q)^x = \sum_{k=0}^\infty q\frac{d}{dq}\binom{x}{k} q^k = \sum_{k=0}^\infty k \binom{x}{k}q^k.$$

Dividing both sides by $(1+q)^{x}$ and setting $q=-p,$ $x=-r$ yields

$$\frac{p\,r}{1-p} = \sum_{k=0}^\infty k \binom{-r}{k} (-1)^k (1-p)^r p^k = \sum_{i=0}^\infty k\,\Pr(k\mid r, p).$$

That is the definition of the expectation, QED.