As I understand it, the CLT states that for a sufficiently large sample size "n", the sampling distribution of the mean from a given population will approximate a normal distribution. Also as the sample size grows, the better the mean approximation becomes. However, when watching a lecture about confidence intervals, the lecturer explains that an assumption when creating confidence intervals is that the population distribution is normally distributed, and if it is not "you should use a large enough sample and let the CLT do the normalization magic for you"; and then proceeds to work off of a single sample. In another video, a different lecturer creates a sampling distribution of the mean and states that we can create a confidence interval from the sampling distribution. So which one is correct and what is the difference. Is the sampling distribution explanation of the CLT only a "proof" of what the theorem dictates and extracting a single large sample is enough? Or am I missing something?
-
2$\begingroup$ You are right to be troubled by this. The CLT is a limit theorem. It is for mathematical statisticians and not for statistical practice. It effectively also assumes that the SD is known, or that the SD estimator is independent of the mean. The latter fails when the distribution is asymmetric. $\endgroup$– Frank HarrellDec 30, 2021 at 13:20
-
2$\begingroup$ @Frank re "not for statistical practice:" I think this reflects an unduly severe (and counterproductive) perspective, perhaps motivated by unstated assumptions about common forms of statistical malpractice. But let's not judge a concept or procedure on the basis of its misapplications! Although the CLT indeed is a limit theorem, one often can perform effective checks to assess whether in a practical finite case that limit is a useful approximation. Moreover, the CLT does not require the analyst to know the SD: the laws of mathematics continue to hold regardless of anyone's mental state! $\endgroup$– whuber ♦Dec 30, 2021 at 15:47
-
$\begingroup$ No, it effectively assumes you know the SD. If you don't you have to estimate it, and the estimate needs to be independent of the mean. With asymmetric distributions, for example, it will not be independent. The CLT assumes that the SD is a good dispersion measure. It often is not. I don't know how to do effective checks in practice with non-huge samples. Bottom line: CI coverage will often be significantly off in at least one tail. Look at performance from a lognormal distribution with $\mu=0, \sigma=1$ for starters. CLT can't fix the "you should have logged but didn't" problem. $\endgroup$– Frank HarrellDec 31, 2021 at 12:34
2 Answers
I'm not sure what your first lecturer was talking about.
In "layman's terms" the CLT says that the errors associated with estimating any parameter using a random sample from any population will follow a normal (or t) distribution.
For example, say you are trying to estimate the true mean of some variable X in a population. You do this by drawing a random sample of observations from the population and taking the mean of X in that. Obviously the mean you see in your sample might be "wrong" (you might have gotten unlucky in your random sample and estimated a value that was higher or lower than the true mean). However, the CLT says that if you kept doing this - drawing zillions of different random samples from the same population and estimating the mean of X in each - the distribution of these estimates will follow a normal distribution (or t distribution of the sample sizes are small) centered around the true value, with a particular standard deviation. And as the sample sizes get larger the standard deviation of this distribution will get smaller (and thus the precision of an estimate from any given sample will rise). This is true regardless of what the distribution of X is in the underlying population. Using this proven mathematical fact, we can quantify how much we can trust that an estimate we get from a random sample is likely to be wrong due to sampling error.
This has enormous real life applications because it's what underlies all forms of probability sampling - election polls, social science research, drug trials, market research, etc. Even though we know that the samples we draw aren't completely random, they're close enough that we can use confidence intervals, and significance tests (which are also based on the CLT) to quantify the minimum amount of uncertainty associated with taking a random sample, rather than collecting data on the whole population.
And this works precisely because it doesn't matter what the distribution of the parameter we're trying to estimate is in the underlying population. The CLT is a statement about how the error associated with random sampling are distributed, regardless of whet it is you are trying to estimate.
-
1$\begingroup$ The CLT is not directly about errors. It is about the sampling distribution of means. And it doesn't follow that you can use it to compute a confidence interval from a sample. The confidence coverage can be arbitrarily bad especially for highly skewed distributions. Most significance tests are not based on the CLT, otherwise the t-test wouldn't have been invented. $\endgroup$ Dec 30, 2021 at 15:24
-
$\begingroup$ I think you were doing very well, until I reached the point "What the CIs tell us ..." Your interpretation, conditional on the hypothetical "if the mean that we estimated ... was the true mean," is unusual and incorrect. $\endgroup$– whuber ♦Dec 30, 2021 at 15:50
-
$\begingroup$ I cut my discussion of CIs, which was wrong because it was trying to make them simpler to understand. But I don't think it's wrong to say that the CLT tells us about how the errors associated with sampling work, or that it's not useful for calculating CIs. I don't think that the distinction between using a z and t distribution for significance tests and CIs is that important for the question asked in the OP. In both cases we are relying on a theorem that says the the distribution of estimates from random samples is symmetric around the true value and "normal-ish." $\endgroup$ Dec 30, 2021 at 16:37
-
$\begingroup$ I agree: the focus on sampling errors is close to the original (historical) motivations behind CIs, too. +1 for the improvements. $\endgroup$– whuber ♦Dec 30, 2021 at 16:45
-
$\begingroup$ please correct me if I'm wrong, so CLT states that regardless of the population distribution the sampling distribution of the mean would be normal. Now we would realistically have access only to one (or a few) samples, but going off of CLT, the larger the sample that we have, the closer the sample mean to population mean, since the sample mean is normally distributed (it's drawn from a normal distribution) so chances are it's near the center of the normal distribution which is the true population mean. Is that correct? If so, what's the difference between CLT and the law of large numbers? $\endgroup$ Dec 30, 2021 at 21:33
The central limit theorem as it's name says is a assymptotic result. It lays out conditions under which we can conclude some very useful properties about properly normalized sums of a increasing set of random variables. A version of the CLT has the following form,
$$\frac{S_n}{s_n}\to^D N(0,1)$$
where $S_n$ is the partial sum of zero-mean random variables (WLOG as we can always subtract the mean if it's finite) and $s_n$ is a normalizing sequence. Under the assumption of independent and identically distributed random variables with mean $\mu$ and standard deviation $\sigma$ we get the common form,
$$ \frac{\sqrt{n}\left(\sum_{k=1}^nX_k - \mu\right)}{\sigma} \to^D N(0,1)$$
Notice that when the sample size is large enough, we get a good pivotal quantity to build confidence intervals, also by the interval-test duality we also get a way to test hypothesis! Search a little about the Wald's test, for example.
There are many other CLTs, some don't assume indepence, but only that the random variables are a Martingale sequence, $$\mathbb{E}[X_n|\mathcal{F}_{m}]= X_m, \; 1 \leq m < n$$ thus the CLT has many forms and extensions to different sets of assumptions.
It's a beautiful theorem that gives us tools to do inference when working with large sample sizes.
