# Understanding the Central Limit Theorem (CLT)

I have a silly, dumb question which you will surely laugh about. But since I believe that so many people think wrong here, I really want to ask it, since I might have something overseen.

My question revolves around the Central Limit Theorem. To read from the same page, I quickly state it here:

Let $X_{1}, ..., X_{n}$ be IID with mean $\mu$ and variance $\sigma^2$. Let $\overline{X_{n}} = n^{-1} \sum_{i=1}^{n} X_{i}$. Then lets define

$Z_{n} := {\frac{\sqrt{n}(\overline{X_{n}}-\mu)}{\sigma}} \rightarrow Z$ where $\rightarrow$ means convergence in distribution, $Z \sim N(0,1)$.

The central limit theorem therefore does the following: We take the random variable $\overline{X_{n}}$ which as one can show: $\mathbf{E}(\overline{X_{n}}) = \mu$ and $Var(\overline{X_{n}}) = \frac{\sigma^2}{n}$. Then we rescale it, as if we would assume it is a normal distributed random variable, because for any normal distributed random variable $Y \sim N(\mu_n, \sigma_n^2)$ one can show that $\frac{Y-\mu_n}{\sigma_n} \sim N(0,1)$. And then, after we rescaled n (the rescaled version is exactly the statement of the CLT as written above), we notice that "by suprise" this scaled version of $\overline{X_{n}}$ is $\sim N(0,1)$. In other words, our assumption that $\overline{X_{n}}$ is normal distributed was right. Why does this suprise us? Because $\overline{X_{n}}$, or better to say the underlying $X_{1}, ..., X_{n}$ come from any distribution. Apparently the normal distribution is for this scaled version of $\overline{X_{n}}$ some kind of a "convergence goal", not considering where it started.

Fair enough. Until here we should be on track. Now here is my issue:

I frequently here from people, especially those who have not deeply studied statistics and use it for example in an application in business, that not just this very specific term $\overline{X_{n}}$ converges in distribution against a standard normal distribution, but also "any sequence" of random variables, as long as you draw enough of them. Let me firstly describe it a little bit intuitively (because that's how I here it from them), and than try to formalize it and understand what this would imply). So what they say is: you draw sufficiently many random variables $A_1, ..., A_n$ from any distribution (so n should be large). Then you plot the histogram of these n random variables that you have just drawn. Then, this histogram looks like a normal distribution (this is what they claim).

So here is what I imply from their statement: If that would be true, than any sufficiently large sequence $A_1, ..., A_n$ would "form" a normal distribution. However, this would mean that at least from a certain value $n_0 > 0$ on, all $A_j$ with $j \geq n_0$ would be normal distributed. But does this make any sense? We know that they follow a different distribution, right? So their theroy is wrong, right?

Other questions related to this are: What can we imply from this very specific statement of the CLT about the average of random variables? Can we imply anything on the random variables themselves? Or some other statements? Is there any theorem that the above statement can be implied from, using CLT?

Thank you guys so much for your help! I would also appreciate any good literature that could help me understand.

• Because I am not sure whether you are referring to this or not, I post this as a comment. AFAIK the central limit theorem applies to the distribution of the means drawn from your data. This would mean (no pun intended) that if your sample $n$ is large enough, the distribution of means you calculate within different 'draws'/samples tends be normally distributed irregardless of the underlying population. E.g. 1000 times calculating the mean within a sample of 100 numbers drawn out of a uniform distribution (==not normal) will give a distribution of means which is approximately normal. – IWS Jan 13 '17 at 8:48
• – Tim Jan 13 '17 at 15:17
• Note that the distribution of the scaled version of $\bar X_n$ is never Normal unless the common underlying distribution of each $X_i$ already is itself Normal. Furthermore, no assumptions whatsoever are made about the distribution of $\bar X_n$ in the CLT, nor does the CLT make any assertions about any particular $\bar X_n$. – whuber Jan 13 '17 at 16:15
• @whuber Good reply! I just want to comment: By the berry esseen theorem, under a further assumption on the third moment, we also have a rate of convergence for the CLT. (This is fortunate since statisticians often use the CLT for finite sample inference.) – user795305 Jan 19 '17 at 17:46

So here is what I imply from their statement: If that would be true, than any sufficiently large sequence $A_1, \cdots,A_n$ would "form" a normal distribution.

It's hard to interpret this. If you want to get a single random variable, then you need some operation to reduce this sequence of many random variables down to one. The classical central limit theorem says that, roughly, if you use the operation "sum", then the result will tend to get more normal in distribution as you add more summands.

Another possible interpretation is that they are saying the joint distribution tends to become a normal random vector, but this is clearly absurd. For example, if we take a sequence of independent coin flips, the joint distribution of all the flips is uniform over all the $2^n$ possibilities. Somehow I doubt this is what they were going for though.

Finally there is the thought that in any sufficiently long sequence of random variables, the ones at the end must eventually be normal. This would mean that there are a finite number of non-normal variables, that we must somehow "use up", and we would only be left with the normal ones. But I built a robot to flip a coin as long as I tell it to. Look, it's there doing it right now. Flip, flip, flip...

As stated, your friends belief is "not even wrong" (to quote Fermi), it just makes no sense.

• The conjecture as described by op must be wrong, there are many well known sequences of random variables which do have limit distribution that is normal – Repmat Jan 13 '17 at 15:44

"I frequently here from people, especially those who have not deeply studied statistics and use it for example in an application in business, that not just this very specific term $\bar X_n$ converges in distribution against a standard normal distribution, but also "any sequence" of random variables, as long as you draw enough of them."

The Central Limit Theorem applies to sums of random variables properly centered and scaled. Of (almost) any distribution indeed. In its basic form, if we have a collection $\{X_1,...,X_n\}$ of random variables, define $S_n = \sum_{i=1}^n X_i$. What usually converges to a Standard Normal is the quantity

$$\frac {S_n -E(S_n)}{\sqrt {\text{Var}(S_n)}} \to_{d}\mathbf N(0,1),\;\; n\to \infty$$

Any "description of an assembly" of random variables that can be mapped to this quantity is a candidate for convergence in distribution to a Standard Normal.

So "any sequence", if with it it is meant "the leading term of the sequence" -it does not satisfy the CLT. If it is meant "any sequence -and then take the sum and then center and scale", it may satisfy the CLT.