Why doesn't the CLT work for $x \sim poisson(\lambda = 1) $?

Question

So we know that a sum of $n$ poissons with parameter $\lambda$ is itself a poisson with $n\lambda$. So hypothetically, one could take $x \sim poisson(\lambda = 1) $ and say it is actually $\sum_1^n x_i \sim poisson(\lambda = 1) $ where each $x_i$ is: $x_i \sim poisson(\lambda = 1/n) $, and take a large n to get CLT to work.

This (obviously) doesn't work. I assume this has something to do with how CLT works "faster" for random variables which are "closer" to normal, and that the smaller lambda is, the more we get a random variable which is mostly 0 and vary rarely something else.

However, what I explained is my intuition. Is there a more formal way to explain why this is the case?

Thanks!

Answer 1

I agree with @whuber that the root of the confusion seems to be replacing the summation asymptotic in CLT with some sort of division in your argument. In CLT we get the fixed distribution $f(x,\lambda)$ then draw $n$ numbers $x_i$ from it and calculate the sum $\bar x_n=\frac{1}{n}\sum_{i=1}^nx_i$. If we keep increasing $n$ then an interesting thing happens: $$\sqrt n (\bar x_n-\mu)\rightarrow\mathcal{N}(0,\sigma^2)$$ where $\mu,\sigma^2$ are mean and the variance of the distribution $f(x)$.

What you're suggesting to do with Poisson is somewhat backwards: instead of summing the variables from a fixed distribution, you want to divide the fixed distribution into ever changing parts. In other words you take a variable $x$ from a fixed distribution $f(x,\lambda)$ then divide it into $x_i$ so that $$\sum_{i=1}^nx_i\equiv x$$

What does CLT say about this process? Nothing. Note, how in CLT we have ever changing $\sqrt n(\bar x_n-\mu)$, and its changing distribution $f_n(x)$ that converges to a fixed distribution $\mathcal{N}(0,\sigma^2)$

In your setup neither the sum $x$ nor its distribution $f(x,\lambda)$ are changing! They're fixed. They're not changing, they're not converging to anything. So, CLT has nothing to say about them.

Also, CLT doesn't say anything about the number of elements in the sum. You can have a sum of 1000 variables from Poisson(0.001) and CLT won't say anything about the sum. All it does say is that if you keep increasing N then at some point this sum will start looking like a normal distribution $\frac{1}{N}\sum_{i=1}^N x_i, x_i\sim Poisson(0.001)$. In fact if N=1,000,000 you'll get the close approximation of normal distribution.

Your intuition is right only about the number of elements in the sum, i.e. than more the starting distribution is different from normal, then more elements you need to sum to get to normal. The more formal (but still informal) way would be by looking at the characteristic function of Poisson: $$\exp(\lambda (\exp(it)-1))$$ If you $\lambda>>1$, you get with the Taylor expansion (wrt $t$) of the nested exponent: $$\approx\exp(i\lambda t-\lambda/2t^2)$$ This is the characteristic function of the normal distribution $\mathcal{N}(\lambda,\lambda^2)$

However, your intuition is not applied correctly: your displacing the summation in CLT with some kind of division messes things up, and renders CLT inapplicable.

Answer 2

The problem with your example is that you are allowing the parameters to change as $n$ changes. The CLT tells you that for a fixed distribution with a finite mean and sd, as $n \rightarrow \infty$,

$\frac {\sum x - \mu} {\sqrt n} \rightarrow_d N(0, \sigma)$,

where $\mu$ and $\sigma$ are from the mean and sd of the distribution of $x$.

Of course, for different distributions (i.e. higher skewed for example), larger $n$'s are required before the approximation derived from this theorem become reasonable. In your example, for $\lambda_m = 1/m$, an $n >> m$ is required before the normal approximation is reasonable.

EDIT

There is discussion about how the CLT does not apply to sums, but rather to standardized sums (i.e. $\sum x_i / \sqrt n$ not $\sum x_i$). In theory, this is of course true: the unstandardized sum will have an undefined distribution in most cases.

However, in practice, you certainly can apply the approximation justified by the CLT to sums! If $F_{\bar x}$ can be approximated by a normal CDF for large $n$, then certainly $F_{\sum x}$ can too, as multiplying by a scalar preserves normality. And you can see this right away in this problem: recall that if $X_i \sim Pois(\lambda)$, then $Y = \sum_{i = 1}^n X_i \sim Pois(n\lambda)$. And we all learned in our upper division probability course that for large $\lambda$, the CDF of a $Pois(\lambda)$ can be approximated quite well by a normal with $\mu = \lambda$, $\sigma^2 = \lambda$. So for any fixed $\lambda$, we can approximate the CDF of $Y \sim Pois(n\lambda)$ fairly well with $\Phi( \frac{y - n\lambda}{\sqrt{n\lambda} })$ for a large enough $n$ if $\lambda > 0$ (approximation can trivially be applied if $\lambda = 0$, but not the calculation of the CDF as I have written it).

While the CLT does not readily apply to sums, the approximation based on the CLT certainly does. I believe this is what the OP was referring to when discussing applying the CLT to the sum.

Answer 3

The question is, I argue, more interesting if thought about more generally, letting the distribution of the parent Poisson depend on $n$, say with parameter $\lambda_n$ and $\lambda_n = 1$ as a special case. I think it's perfectly reasonable to ask why, and how we can understand that, a central limit theorem does not hold for the sum $S_n = \sum_{i=1}^n X_{i,n}$. After all, it's common to apply a CLT even in problems where the distributions of the components of the sum depend on $n$. It's also common to decompose Poisson distributions as the distribution of a sum of Poisson variables, and then apply a CLT.

The key issue as I see it is that your construction implies the distribution of $X_{i, n}$ depends on $n$ in such a way that the parameter of the distribution of $S_n$ does not grow in $n$. If you would instead have taken, for example, $S_n \sim Poi(n)$ and made the same decomposition, the standard CLT would apply. In fact, one can think of many decompositions of a $Poi(\lambda_n)$ distribution that allows for application of a CLT.

The Lindeberg-Feller Central Limit Theorem for triangular arrays is often used to examine convergence of such sums. As you point out, $S_n \sim Poi(1)$ for all $n$, so $S_n$ cannot be asymptotically normal. Still, examining the Lindeberg-Feller condition sheds some light on when decomposing a Poisson into a sum may lead to progress.

A version of the theorem may be found in these notes by Hunter. Let $s_n^2 = \mathrm{Var(S_n)}$. The Lindeberg-Feller condition is that, $\forall \epsilon >0$:

$$ \frac{1}{s_n^2}\sum_{i=1}^n\mathbb E[X_{i,n} - 1/n]^2I(\vert X_{i,n} - 1/n \vert >\epsilon s_n) \to 0,n\to\infty $$

Now, for the case at hand, the variance of the terms in the sum is dying off so quickly in $n$ that $s_n = 1$ for every $n$. For fixed $n$, we also have that the $X_{i,n}$ are iid. Thus, the condition is equivalent to $$ n\mathbb E[X_{1,n} - 1/n]^2I(\vert X_{1,n} - 1/n \vert >\epsilon) \to 0. $$

But, for small $\epsilon$ and large $n$,

\begin{align} n\mathbb E[X_{1,n} - 1/n]^2I(\vert X_{1,n} - 1/n \vert >\epsilon) &>n\epsilon^2P(X_{1,n}>0) \\ &=\epsilon^2n[1 - e^{-1/n}] \\ &= \epsilon^2n[1-(1 - 1/n + o(1/n))] \\ &= \epsilon^2 + o(1), \end{align}

which does not approach zero. Thus, the condition fails to hold. Again, this is as expected since we already know the exact distribution of $S_n$ for every $n$, but going through these calculations gives some indications of why it fails: if the variance didn't die off as quickly in $n$ you could have the condition hold.