3
$\begingroup$

I have a small application of the central limit theorem:

"Take a sample of 40 exponentially distributed random numbers and calculate their mean. Repeat 1000 times, record each measured mean and calculate the mean of those 1000 means."

Here's my Rcode:

nsim   = 1000
nsamp  = 40
lambda = 0.2

clt <- data.frame(m=numeric(nsim), s=numeric(nsim))

for(s in 1:nsim)
{
    clt$v[s] <- mean(rexp(nsamp, lambda))
    clt$s[s] <- sd(rexp(nsamp, lambda))
}   

cat(mean(clt$v), "\n")
    cat(mean(clt$s), "\n")

I can observe that with increasing number of simulations (i.e. increasing the repetitions of the sampling event), the mean of means and mean of variance more closely agree with the theoretical values (1/lambda).

What I wonder is, does the Central-limit theorem only depend on the number of sampling events (1000) or is it just as much dependent on sufficiently large sample size (i.e. 40 in this example)?

$\endgroup$
2
  • 2
    $\begingroup$ What, in you own words, does the CLT actually say? You might enjoy reading some of our higher-voted threads on the CLT. $\endgroup$
    – whuber
    Jan 18, 2015 at 16:34
  • 1
    $\begingroup$ I don't see the Central Limit Theorem here, only the Law of Large Numbers. $\endgroup$ Feb 22, 2017 at 12:27

3 Answers 3

0
$\begingroup$

The CLT is generally considered a theoretical distribution. We don't actually generate 1000 (or 10,000 or 100,000) samples from it. Rather, we imagine what would happen if we were to replicate this experiment 1000 (or, technically, infinite) times.

The R code you post caps it at 1000, probably because by 1000 samples it will demonstrate the properties of the CLT. But, again, the CLT really says what would happen if you repeated it an infinite number of times.

So, to your question:

does the Central-limit theorem only depend on the number of sampling events (1000) or is it just as much dependent on sufficiently large sample size (i.e. 40 in this example)?

It depends on both (kind of). The number of sampling events actually is infinite (though the CLT properties hold with smaller distributions), and as the sample size increases, the more normal (and smaller the variance) the distribution of means becomes.

$\endgroup$
2
  • $\begingroup$ "The CLT is generally considered a theoretical distribution." The CLT is not a theoretical distribution, but the normal distribution is ;) $\endgroup$
    – Galen
    Aug 4, 2023 at 3:46
  • $\begingroup$ "the CLT really says what would happen if you repeated it an infinite number of times." That's a nice intuitive short-hand, popular in stats pedagogy, but that's not really how it works in terms of the pure math. Lookup $(\epsilon,\delta)$-definition of a limit. $\endgroup$
    – Galen
    Aug 4, 2023 at 3:53
-1
$\begingroup$

The central limit theorem states that you should take samples of sufficiently large size (40 in your example), but it does not provide a limit on the minimum number of such samples, i.e., the statement holds regardless of the number of samples. See point #1 in this answer.

$\endgroup$
1
  • 2
    $\begingroup$ CLT nowhere says “sufficiently large sample”, it says $n \to \infty$. See the second point of the answer you refer to. $\endgroup$
    – Tim
    Sep 9, 2022 at 6:17
-1
$\begingroup$

It does not matter for law of large numbers

I observe that with increasing number of simulations (i.e. increasing the repetitions of the sampling event), the mean of means and mean of variance more closely agree with the theoretical values (1/lambda).

Your application and observation of the behaviour of the mean relates more to the law of large numbers than the central limit theorem.

For the mean of means, it is irrelevant how you create the average. If you average all your samples or first average groups does not matter, because the end result is the same.

Example

Say your sample is $1,2,3,4$ then you could group it and compute average of group averages, but the result is the same (if the group's sizes are equal).

$$\frac{(2+3)/2 + (4+1)/2}{2} = \frac{2+3+4+1}{4}$$


Also note that the mean of the estimated standard deviations may approach some value for larger nsim, but that doesn't need to relate to the simulations following approximately a normal distribution.

The behaviour that this value approaches some constant is due to the law of large numbers.

It matters for central limit theorem

The CLT relates to the distribution of a sample average. When you standardize this based on the population mean and deviation, then the final result approaches a Normal distribution under the right circumstances (independence, finite variance, etc.).

If your question would have been about the central limit theorem or about approximations with a normal distribution then the size of the samples matters (not the number of repetitions, although this may have an effect when you make a histogram).

There is already a question about this: Why does increasing the sample size of coin flips not improve the normal curve approximation?

It deals with the issue of the two different sizes (the sample size and the repetitions size), as well as with problems of drawing a histogram.

$\endgroup$
2
  • $\begingroup$ The sample size certainly does relate to the Law of Large Numbers. I think what you mean is that the expected value of the sample does not rely on a large sample? $\endgroup$
    – Cliff AB
    Oct 24, 2023 at 3:12
  • $\begingroup$ @CliffAB yes, in the first part I am talking about the application. The mean of means is computed in that code and the result of that computation doesn't change when the values of nsim and nsamp change while keeping their product the same. $\endgroup$ Oct 24, 2023 at 7:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.