# Central Limit Theorem for Normal Distribution of Negative Binomial

The question is:

Explain why the negative binomial distribution will be approximately normal if the parameter k is large enough. What are the parameters of this normal approximation?

I have previously asked about parts of this question before but wanting to confirm that my thinking is correct on this as it is the answer I am least comfortable with. I have provided all my working below but I am unsure as to whether the variance I have stated for the Normal distribution is correct from my working or if I have missed/dropped an important variable from the calculation.

1.6(c) From the Central Limit Theorem we know that as the number of samples from any distribution increases, it becomes better approximated by a normal distribution. The equation to show this is:
$\Sigma_{i=1}^nX_i\underset{n\rightarrow\infty}\rightarrow\mathcal{N}(n\mu_x,\sigma{^2}_{\Sigma X}=\sigma^2)$
By defining a negative binomial distribution as the sum of k Geometric distributions:
$X=Y_1+Y_2+...+Y_k=\Sigma^k_{i=1}Y_i$
Where $Y_i\sim{Geometric(\pi)}$
Therefore, as k increases, we can restate the Central Limit Theorem as:
$\Sigma_{i=1}^kY_i\underset{k\rightarrow\infty}\rightarrow\mathcal{N}(k\mu_y,\sigma{^2}_{\Sigma X}=\sigma^2)$
As we have shown that the negative binomial distribution X can be represented as a collection of k independent and identically distributed geometric distributions $\Sigma^k_{i=1}Y_i$
$E[X]=k\times E[Y_i]=k\times\mu_y=\dfrac{k}{\pi}$
We also know that the variance of a geometric distribution is given by the following: $Var(Y_i)=\dfrac{(1-\pi)}{\pi^2}$ So, for a negative binomial distribution, as k becomes larger, it can be shown that it is able to be approximated by:
$X\sim\mathcal{N}(\mu=\mu_x,\sigma^2=\dfrac{1-\pi}{\pi^2})$

• I've got the feeling that the bit I have wrong is that the variance of the distribution is actually $Var(X_i) = k\times Var(Y_i)=\dfrac{k(1-\pi)}{\pi^2}$ – Syzorr Aug 8 '15 at 6:41
• Since i.i.d gemetirc distriction, I think you can use the fact that $Var(Y_1+Y_2+...+Y_n)=Var(Y_1)+Var(Y_2)+...+Var(Y_n)=k \frac{1-\pi}{\pi^2}, Y_1, Y_2, ..., Y_n$ have the same variance $\frac{1-\pi}{\pi^2}$. So your comment is correct. – Deep North Aug 8 '15 at 9:20
• sorry, it should be $Y_k$ – Deep North Aug 8 '15 at 9:47
• "Feel free to just say Yay or Nay" ... may be fine as an answer for your purposes but is unacceptable as an answer here. While I sympathize with the position you take (it indicates a genuine desire to do the work yourself), an answer given that way makes it not fit the stackexchange model. Keeping in mind both the restrictions of the self-study tag and the requirements for answers to be informative (at the very least, longer than a sentence, let alone a word), please reframe your question so that it is answerable with an answer that would not contravene SE rules. – Glen_b -Reinstate Monica Aug 8 '15 at 10:18
• Fair enough. Restating now. I'm trying to figure the best way to ask to get guidance as to whether my understanding of the subject material is correct without crossing over in to getting others to do it for me. – Syzorr Aug 8 '15 at 10:47

You also can use CLT directly,one form of CLT states:

$\frac{\sum_{i=1}^nX_i-n\mu}{\sigma\sqrt{n}}\sim N(0,1)=\Rightarrow\sum_{i=1}^nX_i\sim N(n\mu,n\sigma^2)$

Above equations invovle two theorems: The first one is one form CLT

The second related to multivariate normal distribution, but it also apply to 1-dimensional random vector.

$\sum_{i=1}^k Y_i \sim N(\frac{k}{\pi},k\frac{1-\pi}{\pi^2})$

• The first line does not look like a CLT, there's not even a limit involved. – ekvall Aug 8 '15 at 16:36
• see my edited answer. – Deep North Aug 9 '15 at 1:24
• Thanks for that 2nd theorem explaining how you got from N(0,1) to $N(n\mu,n\sigma^2)$. I haven't encountered it yet and that is a good piece of information to have. – Syzorr Aug 9 '15 at 2:41

The Central Limit Theorem makes a limiting-distribution statement for sums of random variables from which sum we have subtracted the sum's expected value, and which we have divided by its standard deviation. Denoting $\sum_{i=1}^kY_i \equiv S_k$ the CLT can be written as

$$\frac {S_k - E(S_k)}{\sqrt {{\rm Var}(S_k)}} \xrightarrow{d} \mathcal N(0,1),\;\;\; {k\rightarrow\infty}$$

Indeed a Negative Binomial ($X$) random variable with parameters $k$ (number of failures before stopping time) and $p$ (probability of success) can be written as the sum of $k$ independent and identically distributed geometric random variables (with $0$ included in the support) with common parameter $1-p$. So $\sum_{i=1}^kY_i \equiv S_k$ in our case is the sum of these $k$ geometric rv's, and $S_k = X$. We have

$$E(Y_i) = \frac {p}{1-p} \implies E(S_k) = \frac {kp}{1-p}$$ $${\rm Var}(Y_i) = \frac {p}{(1-p)^2} \implies {\rm Var}(S_k) = \frac {kp}{(1-p)^2}$$

Plugging these into the CLT expression we have

$$\frac {S_k - E(S_k)}{\sqrt {{\rm Var}(S_k)}} = \frac {X - \frac {kp}{1-p}}{\sqrt {\frac {kp}{(1-p)^2}}} \xrightarrow{d} Z \sim\mathcal N(0,1),\;\;\; {k\rightarrow\infty}$$

Then, approximately for "large $k$" ( and not for $k\rightarrow \infty$) we can write (accepting that the distributional result holds for finite $k$)

$$X \sim_{approx} \left(\sqrt {\frac {kp}{(1-p)^2}}\right)\cdot Z + \frac {kp}{1-p}$$

which by standard properties of scaled and shifted random variables implies that $$X \sim_{approx}\mathcal N \left(\frac {kp}{1-p}, \frac{kp}{(1-p)^2}\right)$$

1.6(c) From the Central Limit Theorem we know that as the number of samples from any distribution increases, it becomes better approximated by a normal distribution.

This is not what the central limit theorem says. The CLT does not hold for every distribution, and in its standard form it concerns properly scaled and standardized sample averages. The statement $\sum_{i=1}^n X_i \underset{n \to \infty}{\to}N(n\mu_x,.)$ is not quite correct, even if we take the mode of convergence to be understood from the context. If $n$ approaches infinity, you cannot have an $n$ left on the right hand side. Indeed, if the $X_i$ are independent and identically distributed geometric random variables, $\sum_{i=1}^nX_i \overset{a.s}{\to} \infty$ so certainly the sum cannot converge in distribution, which is a weaker form of convergence, to something else.

You can save your argument by being more careful with the central limit theorem, however.

• The Central Limit Theorem is for the arithmetic mean so the variant of it I have used here is an approximation of the sum of independent random variables selected from a distribution. As n increases, $\bar{X}$ becomes a better approximation of $\mu_x$. However, we aren't interested in the distribution of the potential mean but the distribution of the sample of X – Syzorr Aug 9 '15 at 21:42
• It's fine to use the CLT to motivate a large sample approximation, but what you write in the section I quoted is incorrect. The fact that you have an $n$ on the right hand side in a limit ought to convince you there is a mistake somewhere. I'm sure you have the right idea, but as it stands what you have written is incorrect, which is what I tried to highlight with this answer. Not sure what you mean with the "distribution of the potential mean"? – ekvall Aug 9 '15 at 21:57