# Is Central Limit Theorem about multiple samples or just one?

I've studied CLT and my understanding is that multiple samples will generate a normal distribution centered in the mean of the population. However, today, one post in Linkedin was saying that "CLT says that a sample big enough have the same characteristics of the population". Is that precise? It doesn't seen to be. Did the author of the post make any mistake or I'm missing something?

• Linkedin may not prove the optimal medium to study probability theory. Commented Apr 6 at 12:19
• @Xi'an for sure! It's just that the statement sounded weird to me, but still close to what I believe, so I wanted to discuss it further. Sorry if my question is too dumb for you, I'll avoid this sort of question in the future. Cheers! Commented Apr 6 at 12:55
• No question is too dumb. What Xi'an wanted to tell is places like LinkedIn aren't reliable. There are tons of good books that discuss such classical results at length for you to consult. Commented Apr 6 at 13:10
• The basic CLT is about independent identically distributed random variables and the limiting distribution of the (rescaled and relocated) sample mean or sum. If sampling from a population is without replacement, then it fails the independent requirement. Commented Apr 7 at 14:59
• Commented Apr 8 at 0:43

First let's be clear about the meaning of the word sample. A sample contains a(n often large) number of (often but not always independent) observations from a population. In some fields of study, the separate observations are called "samples", but in statistics the whole sequence of observations is called the sample.

"CLT says that a sample big enough have the same characteristics of the population"

That is wrong. CLT does not say that. It is true that as the sample size grows, the distribution of the sample approaches that of the population.

But CLT is not about the distribution of the sample; CLT is about the distribution of the mean or the sum of the sample, and says that after standardization, that will approach the standard normal distribution, not the distribution of the population.

• +1, Michael for stressing on the actual meat of various forms of CLT: those deal with the sum or mean. As mentioned in my other comment, GC deals with a.s. uniform convergence of empirical measure -- the confusion stems from reading dubious tutorials or blogs which in the name of oversimplification kill the real meanings of the results. Commented Apr 7 at 3:16

The post sounds like the common but incorrect misconception about the central limit theorem discussed in my question here. The phrasing makes it sound like the central limit theorem means that, as you collect more and more data, the data will have a distribution approaching the population distribution. Under mild assumptions, this is true, but that is the Glivenko-Cantelli theorem, not the central limit theorem.

The danger in mixing together the statements of the central limit theorem and the Glivenko-Cantelli theorem is to incorrectly assume that a large sample will have a Gaussian distribution. The answers to my linked question, particularly the answer I accepted, do a good job of explaining why that is absurd (no matter how common that misconception might be, even among people with generally good training in mathematics, just not this particular topic).

• That's actually very good thing to point out, Dave. However, even if one reads the formal results, they could see the differences. One is talking about the absolute uniform convergence of empirical distribution function , which is different from any version of CLT. The coin flips example debunks the myth. Commented Apr 6 at 15:19
• @User1865345 No argument from me that the two theorems are different! Somehow, though, even people with otherwise good training in mathematical subjects get confused.
– Dave
Commented Apr 6 at 15:54
• I agree, Dave. The main problem is in order to be intuitive and over-simplistic, tutorials or blogs (in OP's case, LinkedIn) make it a mess by diluting these two results turning them into a hodge podge. Read good books! Commented Apr 7 at 3:20

Central limits, simply speaking, talk about weak convergence of distributions of sums of independent random variables to infinitely divisible distributions (cf. $$\rm [I]$$).

The general framework has been thoroughly discussed in $$\rm [II]$$ (this has been used in this post of mine).

Let $$(\Omega, \boldsymbol{\mathfrak A}, \Pr)$$ be a probability space. Consider the sequence of probability spaces $$\langle (\mathcal X_i, \boldsymbol{\mathfrak A}_i, \mathbf P_i)\rangle_{i=1}^\infty,$$ where $$(\mathcal X_i, \Vert \cdot \Vert_i)$$ is a normed linear space and $$\mathbf P_i$$ is the induced probability measure on $$(\mathcal X_i, \boldsymbol{\mathfrak A}_i)$$ with respect to $$\Pr$$ via $$X_i: \Omega\to \mathcal X_i.$$

Let's see how the simplest form of CLT, aka Lindeberg and Lévy, works in this setup:

Take $$\mathcal X_i:=\mathbb R, ~\boldsymbol{\mathfrak A}_i:=\boldsymbol{\mathfrak B}_\mathbb R~\forall ~i\in \mathbb N.$$ Let $$\langle Z_i\rangle_{i=1}^\infty$$ be independent and identically distributed random variables with finite mean $$\mu$$ and variance $$\sigma^2.$$ Define $$X_i:=\sqrt{n}(\bar Z_i -\mu)/ \sigma.$$ Here $$\mathbf P_i$$ is a probability measure on $$(\mathbb R, \boldsymbol{\mathfrak B}_\mathbb R).$$ By CLT, we have $$\lim_{i\to\infty}\mathbf P_i((-\infty, t]) =\Phi(t),$$ where $$\Phi(\cdot)$$ is the standard normal cdf.

## References:

$$\rm [I]$$ Probability and Stochastics, Erhan Çinlar, Springer-Verlag, $$2011,$$ sec. $$3.8,$$ p. $$127.$$

$$\rm[II]$$ Theory of Statistics, Mark J. Schervish, Springer-Verlag, $$1995,$$ sec. $$7.1.2,$$ ex. $$7.5,$$ pp. $$395-398.$$

It sounds to me like the poster on LinkedIn was confusing the Central Limit Theorem with the Law of Large Numbers. This appears to be a common source of confusion.

As per the anonymous LinkedIn user,

"a sample big enough have the same characteristics of the population"

In other words, as sample size increases, the sample mean approaches the true (population) mean. That's the Law of Large Numbers.

• Welcome to Cross Validated! The LinkedIn post makes no mention of the mean or a particular way of estimating the mean (such as the common $\bar X$), and there are many more characteristics of a distribution than its expected value (which might not even exist), which is why I brought up Glivenko-Cantelli.
– Dave
Commented Apr 8 at 20:09
• Good point. "Characteristics" is an unspecific term that doesn't have to refer to the mean in particular.
– Olle
Commented Apr 9 at 7:21