CLT for random sums: Anscombe's Theorem vs. “classical” version

Question

Given a compound Poisson distribution $$S(t):=\sum_{k=1}^{N(t)} X_{k}$$ with

1. $N(t)\in\mathbb{N},\,t\geq0$ a Poisson process with rate $\lambda.$
2. $X_{k}$ are non-negative iid random variables such that $\mathbb{E}\left[X_{k}\right]<\infty$ and $\sigma^{2}:=\operatorname{Var}(X_{k})<\infty$.
3. $N$ and $(X_{1},X_{2},\ldots)$ are independent.

Then $$\mathbb{E}\left[ S\right]= \mathbb{E}\left[ N\right]\mathbb{E}\left[ X_{1}\right]$$ as well as $$\operatorname{Var}\left[ S\right]= \operatorname{Var}\left[ N\right]\,\mathbb{E}\left[ X_{1}\right]^{2} + \mathbb{E}\left[ N\right]\operatorname{Var}\left[ X_{1}\right].$$ Then we have, by a version of the central limit theorem, that

$$\frac{S(t)-\mathbb{E}\left[S(t) \right]}{\sqrt{\operatorname{Var}\left[S(t) \right]}}\stackrel{d}{\to} \mathcal{N}(0,1)\,\text{as } t\to\infty.$$

Suppose that $\mathbb{E}\left[ X_{k}\right]=0$ and $N(t), t \geq 0$ is a family of positive, integer valued random variables, such that there is a $\theta >0$ $$ \frac{N(t)}{t}\stackrel{\mathbb{P}}{\rightarrow}\theta,\,\text{ as } t\to\infty. $$

According to Renyi's or Anscombe's Theorem, we then have \begin{align*} \frac{S(t)}{\sigma\sqrt{N(t)}} \xrightarrow[]{d} \mathcal{N}(0,1)\,\text{ as } t\to\infty \\ \frac{S(t)}{\sigma\sqrt{\lambda\cdot t}} \xrightarrow[]{d} \mathcal{N}(0,1)\,\text{ as } t\to\infty, \\ \end{align*} which is different from the above normal approximation (using Wald's identity).

My questions are now:

1. What is the key difference between the two approximations? And which one is when preferrable?

2. Given a realization of a compound Poisson process $$ s=\sum_{k=1}^{N} x_{k},$$ with unknown parameters. How can one estimate the parameters in order to afterwards apply the central limit theorem (with Wald's identity)?

For example, if one uses sample means, we would get $$\frac{s-N\cdot \frac{\sum_{k=1}^{N}x_{k}}{N}}{\tilde{\sigma}}=\frac{s-s}{\tilde{\sigma}}=0,$$ which is not useful.

---

---

$\textbf{Update}$: Thanks alot for the answer so far, which brings me to a subsequent question. Suppose that we do not require $X_{k}$ to be non-negative in the second condition, but that $X_{k}\in L^{2}$ with mean $\mu_{X}$ and variance $\sigma_{X}^{2}$.

We then have by the CLT for compound Poisson processes that $$\frac{S(t)-\mu_{X}\lambda t}{\sqrt{\lambda t(\mu_{X}^{2}+\sigma_{X_{k}}^{2})}}= \frac{S(t)-\mu_{X}\lambda t}{\sqrt{\lambda t\mathbb{E}\left[X_{k}^{2} \right]}}\stackrel{d}{\to} \mathcal{N}(0,1)\,\text{as } t\to\infty.$$

On the other hand, center $X_{k}$ around its mean, i.e. $Y_{k}:=X_{k}-\mu_{X}$. Then $\mathbb{E}\left[Y_{k}\right]=0$ and $\operatorname{Var}(Y_{k})=\operatorname{Var}(X_{k})=\sigma_{X}^{2}$.

By applying Anscombe's Theorem on $\sum_{k=1}^{N(t)} Y_{k}$, we obtain \begin{align*} \frac{\sum_{k=1}^{N(t)}X_{k} - N(t)\mu_{X}}{\sigma_{X}\sqrt{N(t)}} = \frac{S(t) - N(t)\mu_{X}}{\sigma_{X}\sqrt{N(t)}}\xrightarrow[]{d} \mathcal{N}(0,1)\,\text{ as } t\to\infty. \end{align*} As $\frac{N(t)}{t}\to \lambda$ almost surely, we obtain \begin{align*} \frac{S(t)-\mu_{X}\lambda t}{\sqrt{\lambda t\sigma_{X}^{2}}}=\frac{S(t)-\mu_{X}\lambda t}{\sqrt{\lambda t\left(\mathbb{E}\left[X_{k}^{2} \right]-\mathbb{E}\left[X_{k}\right]^{2} \right)}}\xrightarrow[]{d} \mathcal{N}(0,1)\,\text{ as } t\to\infty, \end{align*} which differs by $-\mathbb{E}\left[X_{k}\right]^{2}$ to the previous expression. This seems not compatible, yet I am not sure where the mistake lies.

Answer 1

First, the two approximations are not different, although they do appear on the surface to be.

In the initial expression for $\text{Var}[S]$, note that:

1. $\mathbb{E}[X_1]^2 = 0$ by assumption in the run-up to your application of Anscombe's Theorem,

2. $\mathbb{E}[N]$ is really $\mathbb{E}[N(t)]$, which equals $\lambda t$, and

3. $\text{Var}[X_1] = \sigma^2$.

Substituting into the r.h.s. of the expression for $\text{Var}[S]$ results in $\text{Var}[S] = \sigma^2\lambda t$, which matches the square of the denominator of the left hand side of the Anscombe-derived expressions.

Second, if you only have one realization $s$ of the compound Poisson process, this means you have a sample of size 1. Naturally, this will doom you in any situation where you have to estimate, say, a mean and a standard deviation.

If, on the other hand, you have observed the components of the (one realization of the) process, i.e., $N$ and the individual $x_k$, you are better off, thanks to that assumption that $N$ is distributed Poisson, which means that a single observation allows you to estimate the variance as well as the mean. Let us assume without loss of generality that you have observed one time period ($t=1$); then $\hat{\lambda} = N$, and you can estimate $\mathbb{E}[X_1]$ and $\sigma^2$ using the usual estimators $\bar{x}$ and $s^2_x$ - as long as $N > 1$. In this case, the sample mean of the process will naturally equal your single observed value of $s$, and your sample variance will equal $\hat{\lambda}s^2_x$ (under a null hypothesis that $\mathbb{E}[X_1] = 0$.) If $N=1$, you are again stuck, as you can't estimate the variance of the $x_k$.