Is applying the CLT to the sum of random variables a good approximation?

Question

I use $(\mu, \sigma^2)$ to mean a distribution with mean $\mu$ and variance $\sigma^2$, $\mathcal{N}$ added to mean the normal distribution.

Let's suppose $X_1, \dots, X_n\overset{\text{iid}}{\sim}(\mu, \sigma^2)$ with $\sigma^2 < \infty$. The formal statement of the central limit theorem (CLT) says that $$\dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}\overset{d}{\to}\mathcal{N}(0, 1)\text{.}$$ It's discussed here that the statement $$\bar{X}_n \sim \mathcal{N}(\mu, \sigma^2/n)$$ is not a statement about convergence in distribution, but rather, an approximation. This approximation is frequently cited as being a pretty decent approximation when $n \geq 30$.

Now, theoretically, we could go one step further and say that $$\sum_{i=1}^{n}X_i\sim\mathcal{N}(n\mu, n\sigma^2)\tag{1}$$ is an approximate statement from the CLT.

Given that $(1)$ isn't the actual CLT, I wonder how well this approximation performs. Does it perform well in general? Honestly, I'd be concerned about this in the case of a particularly skewed distribution.

If this is too broad, I can close this.

Answer 1

Going the other way around, if the Z-score were truly a standard normal distribution, then your subsequent approximations would be exact. The degree of error should roughly scale with some measure of distance between the Z-score distribution and the standard Gaussian.

We can use K-S distance as our metric in the space of CDFs. Let's say that we will collect $N$ samples and our (unknown) true sample CDF of the Z-score of these $N$ samples will have a K-S distance of $\epsilon_N$: $\max_z |F_{Z_n}(z) - F_{\Phi}(z)| = \epsilon_N$.

Now, going from the $F_{Z_n}(z)$ to $F_{S_n}(s)$ where $S_n = \sum_1^N X_i$ involves only a shift of scale and location (i.e., a linear transformation $Lz$ of the argument of $F_{Z_n}(z)$). The same applies to get $F_{\Phi}(z)$ to a sum of normal random variables with the same mean and variance as your actual population. In fact, you will be making the exact same transformation to both variables, so we will simply be mapping $F_{Z_n}(z) \mapsto F_{Z_n}(L^{-1}z)$ and similarly for $F_{\Phi}$ -- because we are subjecting each distribution's argument to the same transformation, we will preserve vertical distances.

So, the KS distance for the $F_{S_n}$ will converge to zero at the same rate as for $F_{Z_n}$. However, $F_{S_n}$ doesn't have a limiting distribution (it's basically $F(x)=0.5$, which is not a distribution) whereas $F_{Z_n}$ converges to an actual distribution function.