# Cental limit theorem for average of iid variables

The Wikipedia entry on the CLT states at one point: "For fixed large $n$ one can also say that the distribution of $S_n$ is close to the normal distribution with mean $\mu$ and variance $\frac1n\sigma^2$." $S_n = (\sum_{i=1}^nX_n) / n$ and $X_i$ are iids with mean $\mu$ and variance $\sigma^2$.

I don't quite see how this follows from the other, more formal definitions. Is this statement true and what is the source/proof?

-
Which part is unclear? The moments of $S_n$? How the approximate normality of $S_n$ follows from the approximate normality of $\sum_{i=1}^nX_n$? –  MånsT Sep 6 '12 at 8:14
@MånsT Well all formulation seem to revolve around a scaled distribution: $\sqrt(n)((\sum_(i=1)^n X_n)/n - \mu) \rightarrow N(0, \sigma^2)$. How do we get to the form in the original question from there? –  Lubo Antonov Sep 6 '12 at 8:17

You seem to be asking about how the rescaling works. It utilizes the fact that linear transformations of a normal random variable are normal: if $$X\sim {\rm N}(m,s^2)$$ and $Y=\frac{X}{b}+a$ then $$Y\sim {\rm N}\Big(\frac{m}{b}+a,\frac{s^2}{b^2}\Big).$$

Now, if $$Z=\sqrt{n}\Big(\frac{1}{n}\sum_{i=1}^nX_n-\mu\Big)\approx {\rm N}(0,\sigma^2)$$ then consequently $$S_n=\frac{1}{n}\sum_{i=1}^nX_n=\frac{1}{\sqrt{n}}Z+\mu\approx{\rm N}\Big(\mu,\frac{\sigma^2}{n}\Big).$$ This corresponds to $m=0$, $s=\sigma$, $a=\mu$ and $b=\sqrt{n}$ in the general formulae above.

-
Thanks, this is exactly what I had trouble with! –  Lubo Antonov Sep 6 '12 at 9:25
When you have a limit, you don't automatically know that you are actually close for any particular fixed $n$. An additional result like that is called effective. One effective version of the Central Limit Theorem is the Berry-Esseen theorem, which bounds the difference between the CDF of $S_n$ and $\Phi$ in terms of the third central moment of $X$ and the standard deviation of $X$. To use this, you need the third central moment to exist, and you need a bound on it. The Central Limit Theorem does not require the third central moment to exist, though, so this does not work every time the Central Limit Theorem applies.
In addition, since you don't have a uniform bound on the third central moments even when they exist, you can't say that $S_n$ resembles a normal distribution well. More precisely, if you choose $n$, I can pick a distribution for $X$ so that $S_n$ is far from normal, so that the convergence happens later. Let $X$ be $0$ with high probability ($\frac{n^2-1}{n^2}$), with equal small ($\frac{1}{2n^2}$) chances to be $n$ and $-n.$ This has mean $0$ and variance $1$, but the probability is over $\frac{n-1}{n}$ that $S_n = 0$.
So if I understand correctly, you are saying that "for fixed large $n$" makes the statement inaccurate, and it should be "without bound", as the Berry-Essen entry actually states. If I have a distribution with a large unknown, but definitely finite, $n$ (i.e. a non-theoretical physical phenomenon with a given distribution), can I use the Normal approximation, or not? –  Lubo Antonov Sep 6 '12 at 8:38
For any finite $n$, the normal approximation will have an error. Is the error small enough for you, or not? Well, if you have the additional information the Berry-Esseen theorem requires, then you can bound the error, and decide whether the error is small enough. If you don't know, you could use the normal approximation anyway, and maybe it will work and maybe it won't. There are heuristics people use for deciding whether to use a normal approximation, such as that it works better near the center, and with $n \gt 30$. Whether those are good enough depends on $X$ and your error tolerance. –  Douglas Zare Sep 6 '12 at 9:39