Central Limit Theorem for Normal Distribution of Negative Binomial

Question

The question is:

Explain why the negative binomial distribution will be approximately normal if the parameter k is large enough. What are the parameters of this normal approximation?

I have previously asked about parts of this question before but wanting to confirm that my thinking is correct on this as it is the answer I am least comfortable with. I have provided all my working below but I am unsure as to whether the variance I have stated for the Normal distribution is correct from my working or if I have missed/dropped an important variable from the calculation.

1.6(c) From the Central Limit Theorem we know that as the number of samples from any distribution increases, it becomes better approximated by a normal distribution. The equation to show this is:
$\Sigma_{i=1}^nX_i\underset{n\rightarrow\infty}\rightarrow\mathcal{N}(n\mu_x,\sigma{^2}_{\Sigma X}=\sigma^2)$
By defining a negative binomial distribution as the sum of k Geometric distributions:
$X=Y_1+Y_2+...+Y_k=\Sigma^k_{i=1}Y_i$
Where $Y_i\sim{Geometric(\pi)}$
Therefore, as k increases, we can restate the Central Limit Theorem as:
$\Sigma_{i=1}^kY_i\underset{k\rightarrow\infty}\rightarrow\mathcal{N}(k\mu_y,\sigma{^2}_{\Sigma X}=\sigma^2)$
As we have shown that the negative binomial distribution X can be represented as a collection of k independent and identically distributed geometric distributions $\Sigma^k_{i=1}Y_i$
$E[X]=k\times E[Y_i]=k\times\mu_y=\dfrac{k}{\pi}$
We also know that the variance of a geometric distribution is given by the following: $Var(Y_i)=\dfrac{(1-\pi)}{\pi^2}$ So, for a negative binomial distribution, as k becomes larger, it can be shown that it is able to be approximated by:
$X\sim\mathcal{N}(\mu=\mu_x,\sigma^2=\dfrac{1-\pi}{\pi^2})$

Answer 1

You also can use CLT directly,one form of CLT states:

$\frac{\sum_{i=1}^nX_i-n\mu}{\sigma\sqrt{n}}\sim N(0,1)=\Rightarrow\sum_{i=1}^nX_i\sim N(n\mu,n\sigma^2)$

Above equations invovle two theorems: The first one is one form CLT

The second related to multivariate normal distribution, but it also apply to 1-dimensional random vector.

For your case:

$\sum_{i=1}^k Y_i \sim N(\frac{k}{\pi},k\frac{1-\pi}{\pi^2})$

Answer 2

1.6(c) From the Central Limit Theorem we know that as the number of samples from any distribution increases, it becomes better approximated by a normal distribution.

This is not what the central limit theorem says. The CLT does not hold for every distribution, and in its standard form it concerns properly scaled and standardized sample averages. The statement $\sum_{i=1}^n X_i \underset{n \to \infty}{\to}N(n\mu_x,.)$ is not quite correct, even if we take the mode of convergence to be understood from the context. If $n$ approaches infinity, you cannot have an $n$ left on the right hand side. Indeed, if the $X_i$ are independent and identically distributed geometric random variables, $\sum_{i=1}^nX_i \overset{a.s}{\to} \infty$ so certainly the sum cannot converge in distribution, which is a weaker form of convergence, to something else.

You can save your argument by being more careful with the central limit theorem, however.

Answer 3

The Central Limit Theorem makes a limiting-distribution statement for sums of random variables from which sum we have subtracted the sum's expected value, and which we have divided by its standard deviation. Denoting $\sum_{i=1}^kY_i \equiv S_k$ the CLT can be written as

$$\frac {S_k - E(S_k)}{\sqrt {{\rm Var}(S_k)}} \xrightarrow{d} \mathcal N(0,1),\;\;\; {k\rightarrow\infty} $$

Indeed a Negative Binomial ($X$) random variable with parameters $k$ (number of failures before stopping time) and $p$ (probability of success) can be written as the sum of $k$ independent and identically distributed geometric random variables (with $0$ included in the support) with common parameter $1-p$. So $\sum_{i=1}^kY_i \equiv S_k$ in our case is the sum of these $k$ geometric rv's, and $S_k = X$. We have

$$E(Y_i) = \frac {p}{1-p} \implies E(S_k) = \frac {kp}{1-p}$$ $${\rm Var}(Y_i) = \frac {p}{(1-p)^2} \implies {\rm Var}(S_k) = \frac {kp}{(1-p)^2}$$

Plugging these into the CLT expression we have

$$\frac {S_k - E(S_k)}{\sqrt {{\rm Var}(S_k)}} = \frac {X - \frac {kp}{1-p}}{\sqrt {\frac {kp}{(1-p)^2}}} \xrightarrow{d} Z \sim\mathcal N(0,1),\;\;\; {k\rightarrow\infty}$$

Then, approximately for "large $k$" ( and not for $k\rightarrow \infty$) we can write (accepting that the distributional result holds for finite $k$)

$$X \sim_{approx} \left(\sqrt {\frac {kp}{(1-p)^2}}\right)\cdot Z + \frac {kp}{1-p}$$

which by standard properties of scaled and shifted random variables implies that $$X \sim_{approx}\mathcal N \left(\frac {kp}{1-p}, \frac{kp}{(1-p)^2}\right)$$