# Does the Central Limit Theorem Apply to All Finite Samples Even If They Come From Distributions That Don't Have a Finite Variance?

Some distributions, like the Cauchy distribution, don't have a finite variance, and therefore the central limit theorem does not apply to them. If I have a thousand randomly selected observations from a Cauchy distribution, though, I have a finite sample with a finite mean and variance. Since I can calculate a mean and a variance of these observations, I would imagine the central limit theorem would apply. Does the central limit theorem always apply to finite samples regardless of what distribution they come from? Are there any instances where the central limit theorem wouldn't apply to a finite sample, and if there are, what are they?

Here's some R code to show what I mean.

Values <- rcauchy(1000)
Sample_Size <- c(1, 10, 100)
Number_of_Iterations <- 1000
Sample_Means <- sapply(seq_len(length(Sample_Size)), function (x) {
sapply(seq_len(Number_of_Iterations), function (y) {
mean(sample(Values, Sample_Size[x]))
})
})
par(mfrow = c(1, length(Sample_Size)))
mapply(function (x, y) {
hist(Sample_Means[, x], xlab = 'Value', main = paste('Sample Size =', y))
shapiro.test(Sample_Means[, x])
}, x = seq_len(ncol(Sample_Means)), y = Sample_Size, SIMPLIFY = F)

• The central limit theorem concerns random variables. What are your random variables, and what distributions do they have?
– Dave
Commented Jun 20, 2023 at 14:14
• Unless your population includes infinite values, it has a variance; and therefore--as the CLT shows--the standardized means of repeated samples (with replacement!) converge in distribution to the Normal.
– whuber
Commented Jun 20, 2023 at 15:54
• If you have a finite population, so a bounded distribution implying a finite variance, and sample from it with replacement, then the central limit theorem applies. But a Cauchy distribution does not have a finite variance (even if all values are finite) so the central limit theorem does not apply to it. (You could resample with replacement from a particular sample originally from a Cauchy distribution, but the original sample would not have a Cauchy distribution and the original sample's finite variance could vary wildly, so this would not count. Commented Jun 20, 2023 at 17:15
• I edited my question to address your comments - thanks Commented Jun 20, 2023 at 20:17
• I presume you're asking about whether the CLT holds for the mean of a sample from a finite population. The case @Henry mentioned, sampling with replacement, is easier. For the more common case of sampling without replacement, though, CLT's also exist. This paper cites a few and develops a new approach specifically for cases involving causal inference: arxiv.org/pdf/1610.04821.pdf If my memory serves me right this was recently published in JASA. Commented Aug 31, 2023 at 14:53

You can certainly take a set of 1000 observations from a Cauchy distribution, and they will have a finite mean and finite variance

> x<-rcauchy(1000)
> mean(x)
[1] -0.7734487
> var(x)
[1] 435.5169


If you want a Central Limit Theorem, though, you care about the distribution over different samples of size 1000 from the distribution

> x<-rcauchy(1000)
> mean(x)
[1] 0.1649075
> var(x)
[1] 481.9081


These have different finite means and variances. The question is whether the distribution of the mean, across these samples, is Normal. It isn't.

lots.of.means<-replicate(10000, mean(rcauchy(1000)))
qqnorm(lots.of.means)


So, what's happening? Most of the time, as you add observations to a Cauchy distribution, the sample average moves a little bit towards the centre of the distribution. But sometimes you get a really big observation that kicks the sample mean away from the centre of the distribution

x<-rcauchy(1e6)
means<-cumsum(x)/(1:1e6)


If you take a sample of size 1000, most of the time you get a reasonable value, but occasionally there has been a big jump just before $$n=1000$$ and you get an outlier. This never stops happening, no matter how large $$n$$ is, and that's what we mean when we say say the Cauchy distribution doesn't satisfy a central limit theorem or even a law of large numbers.

If you look at the variance of samples it's a bit different

vars<-sapply(1:1e4, function(n) var(x[1:(100*n)]))


Again, the variance mostly goes down as if it's trying to converge to a 'true value', but keeps getting kicked up. Here, the jumps get bigger and bigger as $$n$$ increases, so the sample variance tends to get bigger and bigger -- it diverges to infinity.

So, that's how you can get infinite variance and no mean and no asymptotic normality even though most individual values from a Cauchy distribution look perfectly reasonable. There are enough outliers to mess things up, enough so that you don't get the mean converging no matter how many observations you take.

• Thanks for the thoughtful response. Perhaps my question wasn’t clear, but I was actually wondering about a given finite sample from a Cauchy distribution. In your answer, you continue to generate new (different) finite samples. I’m specifically wondering if the central limit theorem applies to only one finite sample. Does that make sense? Commented Jun 24, 2023 at 21:56
• The R code in my original question helps illustrate my point Commented Jun 24, 2023 at 22:34
• The CLT does not apply to a finite sample at all. It is a result about a particular limit as the sample size goes to infinity, not a result that applies to any specific sample or sample size. That is why the answer above talks about "as $n$ increases" and "no matter how large $n$ is." Commented Jun 25, 2023 at 22:27