Central Limit Theorem: Sample Size or Number of Samples? [duplicate]

Question

The central limit theorem states that if we take a take a large enough sum of random variables, the sum will approach a normal distribution. I am confused about why we focus only on the sample size and not the number of samples. The way I'm thinking about it, when we take a sample of random variables then calculate the sum, that sum is just a singular number (we "compress" the sample into one statistic). If this is correct, then don't we need more samples to get more sums so we can actually plot the distribution? If we only calculate one sum (say with a sample size of 100,000 random variables), then we only have one number which is clearly not enough to create a distribution.

Answer 1

the CLT is just telling you how the average of the sample is distributed. Just as if I tell you the throws of a die are distributed equally with probability 1/6, or that a particular coin is fair (50% heads, 50% tails)

how you check this is a different issue eg by throwing the die multiple times and creating a histogram.

Answer 2

Actually, in practice, when we conduct a survey/experiment. It is costly and takes time to reconstruct the same thing with the same controlled factors many times. Therefore, conducting one time with a large enough sample size will produce the desired results, where we can test the hypothesis under the CLT. Furthermore, some hypotheses based on observed data (eg economic and social data) cannot be reproduced. So, the sample size will give us more advantages than the number of samples.

Answer 3

This question touches on something rather deep and difficult, which isn't clearly discussed in much literature.

The CLT concerns the distribution of the mean of random variables $X_1,\ldots,X_n$ modelled as independently and identically distributed (i.i.d.), for $n\to\infty$. Using the CLT as an approximation for fixed $n$, we may have $n$ observations $x_1,\ldots,x_n$ from which we can compute the mean, and the CLT then allows us to compute tests or confidence intervals regarding the true underlying expected value (somewhat confusingly often also called "mean").

Given that the CLT regards the distribution of the mean and not just the one mean that we can compute from our data, indeed the question is legitimate how we could "observe" this distribution. Well actually, the OP is right that based on the sample of size $n$ we have just observed, we can't do this. We have, so to say, only a single "observation" of the whole sequence of length $n$. The distribution is a distribution over the means of an ideally infinite number of samples of size $n$, which in this situation (which is the standard situation in practice) we don't have at our disposal. In a practical situation, unless arbitrarily more observations could be collected, the distribution of the sample mean of size $n$ is an artificial construct, a "fiction", so to say.

What we however can do, given that we are willing to make a model assumption, is to generate lots of samples of size $n$ from the model, compute their means, and plot the sample distribution of the mean, which will usually look very "normal" if $n$ is large enough (although one could come up with models where a very large $n$ is needed for reaching approximate normality, and for some distributions such as the Cauchy it won't work at all, due to non-existing variances).

These are not real data though, they are produced by the statistician in order to get an idea how the CLT actually works, in what kind of situation, depending on $n$.

An alternative is the nonparametric bootstrap, which consists of generating artificial new samples drawn with replacement from the actually existing real sample of size $n$. This will also make it possible to generate and plot a distribution of sample means. But once more this is artificial (even though based on the existing data) and not always reliable. Both of these approaches though can at least nicely illustrate what the CLT says (or doesn't say) on finite samples, even though they do not deliver the "true" distribution of sample means in the real situation, which would require an infinite amount of real samples of size $n$, which we don't have.

The subtle and deep issue here is that according to the frequentist concept of probability, probabilities of events generally are defined by what happens if the experiment is infinitely repeated. The assumption of "i.i.d." is defined within a probability model (for a collection/sample of random variables), which means that stating that $X_1,\ldots,X_n$ are i.i.d., random variables requires the idea of repetition of the whole sequence $X_1,\ldots,X_n$. And technically, the repetition required to define probability is different from an i.i.d. repetition, as an i.i.d. repetition requires that the i.i.d. model is defined as probability model, implying that the whole thing is to be repeated infinitely. So there are two different kinds of repetition required here, one modelled as i.i.d, and another one defining what the i.i.d. in the concrete situation refers to. The CLT makes probability statements about the distribution of an infinite sequence of i.i.d. random variables, which can only refer to an infinite repetition of the whole infinite sequence! In practice, of course, such a thing does not exist. It's all idealisation. Let's hope with George Box that these models are at least useful, though "wrong". ;-)

Answer 4

The central limit theorem (CLT) tells us what would happen if we drew a large number of samples (of a given size) from the same population. In practice, of course, we usually only draw a single sample. But even in this case, the CLT is useful because it can tell us something about what sort of properties we can expect from the one sample we did.

For example, let's say we're drawing a sample of size N to run an randomized experiment to test whether getting some treatment significantly improves people's score on a test, which could run from 0 to 100.

We do this and we find that the people who got the treatment scored 10 points higher than the people who did not. Does that mean the treatment worked? Not necessarily, because we might have just gotten unlucky and drew a sample where the people who got the treatment happened to have a higher score anyways.

To sort this out we consult the CLT, telling it the sample size we used and the standard deviation of the result. It tells us the following:

"If in reality the treatment had no effect at all, and if you had drawn a zillion different random samples of size N from this same population and done this experiment on all of them, then in only .01% of those samples would you observe a different between the treatment and control groups as large as the one you actually see." (this is a "p value" by the way)

This seems like pretty compelling evidence that the effect we're seeing in the data is not due to us being unlucky in drawing the sample. A pretty useful thing to know!

So you are right that all the CLT can ever tell us is what would happen if we drew a lot of random samples of a given size, we can use this information to learn something about the reliability of the results that we got by drawing only a single random sample. This is why the CLT is so useful, even if we rarely ever draw multiple samples.