Understanding the Central Limit Theorem (CLT)

Question

I have a silly, dumb question which you will surely laugh about. But since I believe that so many people think wrong here, I really want to ask it, since I might have something overseen.

My question revolves around the Central Limit Theorem. To read from the same page, I quickly state it here:

Let $X_{1}, ..., X_{n}$ be IID with mean $\mu$ and variance $\sigma^2$. Let $\overline{X_{n}} = n^{-1} \sum_{i=1}^{n} X_{i}$. Then lets define

$Z_{n} := {\frac{\sqrt{n}(\overline{X_{n}}-\mu)}{\sigma}} \rightarrow Z$ where $\rightarrow$ means convergence in distribution, $Z \sim N(0,1)$.

The central limit theorem therefore does the following: We take the random variable $\overline{X_{n}}$ which as one can show: $\mathbf{E}(\overline{X_{n}}) = \mu$ and $Var(\overline{X_{n}}) = \frac{\sigma^2}{n}$. Then we rescale it, as if we would assume it is a normal distributed random variable, because for any normal distributed random variable $Y \sim N(\mu_n, \sigma_n^2)$ one can show that $\frac{Y-\mu_n}{\sigma_n} \sim N(0,1)$. And then, after we rescaled n (the rescaled version is exactly the statement of the CLT as written above), we notice that "by suprise" this scaled version of $\overline{X_{n}}$ is $\sim N(0,1)$. In other words, our assumption that $\overline{X_{n}}$ is normal distributed was right. Why does this suprise us? Because $\overline{X_{n}}$, or better to say the underlying $X_{1}, ..., X_{n}$ come from any distribution. Apparently the normal distribution is for this scaled version of $\overline{X_{n}}$ some kind of a "convergence goal", not considering where it started.

Fair enough. Until here we should be on track. Now here is my issue:

I frequently here from people, especially those who have not deeply studied statistics and use it for example in an application in business, that not just this very specific term $\overline{X_{n}}$ converges in distribution against a standard normal distribution, but also "any sequence" of random variables, as long as you draw enough of them. Let me firstly describe it a little bit intuitively (because that's how I here it from them), and than try to formalize it and understand what this would imply). So what they say is: you draw sufficiently many random variables $A_1, ..., A_n$ from any distribution (so n should be large). Then you plot the histogram of these n random variables that you have just drawn. Then, this histogram looks like a normal distribution (this is what they claim).

So here is what I imply from their statement: If that would be true, than any sufficiently large sequence $A_1, ..., A_n$ would "form" a normal distribution. However, this would mean that at least from a certain value $n_0 > 0$ on, all $A_j$ with $j \geq n_0$ would be normal distributed. But does this make any sense? We know that they follow a different distribution, right? So their theroy is wrong, right?

Other questions related to this are: What can we imply from this very specific statement of the CLT about the average of random variables? Can we imply anything on the random variables themselves? Or some other statements? Is there any theorem that the above statement can be implied from, using CLT?

Thank you guys so much for your help! I would also appreciate any good literature that could help me understand.

Answer 1

So here is what I imply from their statement: If that would be true, than any sufficiently large sequence $A_1, \cdots,A_n$ would "form" a normal distribution.

It's hard to interpret this. If you want to get a single random variable, then you need some operation to reduce this sequence of many random variables down to one. The classical central limit theorem says that, roughly, if you use the operation "sum", then the result will tend to get more normal in distribution as you add more summands.

Another possible interpretation is that they are saying the joint distribution tends to become a normal random vector, but this is clearly absurd. For example, if we take a sequence of independent coin flips, the joint distribution of all the flips is uniform over all the $2^n$ possibilities. Somehow I doubt this is what they were going for though.

Finally there is the thought that in any sufficiently long sequence of random variables, the ones at the end must eventually be normal. This would mean that there are a finite number of non-normal variables, that we must somehow "use up", and we would only be left with the normal ones. But I built a robot to flip a coin as long as I tell it to. Look, it's there doing it right now. Flip, flip, flip...

As stated, your friends belief is "not even wrong" (to quote Fermi), it just makes no sense.

Answer 2

"I frequently here from people, especially those who have not deeply studied statistics and use it for example in an application in business, that not just this very specific term $\bar X_n$ converges in distribution against a standard normal distribution, but also "any sequence" of random variables, as long as you draw enough of them."

The Central Limit Theorem applies to sums of random variables properly centered and scaled. Of (almost) any distribution indeed. In its basic form, if we have a collection $\{X_1,...,X_n\}$ of random variables, define $S_n = \sum_{i=1}^n X_i$. What usually converges to a Standard Normal is the quantity

$$\frac {S_n -E(S_n)}{\sqrt {\text{Var}(S_n)}} \to_{d}\mathbf N(0,1),\;\; n\to \infty$$

Any "description of an assembly" of random variables that can be mapped to this quantity is a candidate for convergence in distribution to a Standard Normal.

So "any sequence", if with it it is meant "the leading term of the sequence" -it does not satisfy the CLT. If it is meant "any sequence -and then take the sum and then center and scale", it may satisfy the CLT.