To what what sample size does the CLT refer? The size of the random draw (the sample) or the repetitions in a (hypothetical) simulation? [duplicate]

Question

Assume the following population:

If I draw many repeated random samples of size N from this population, their means should be distributed normally - according to the CLT. Here, I draw 100,000 samples of size N from the distribution above and calculate the mean for each of the samples. Why does this convergence to the normal work better if the individual sample sizes are larger? See below:

My guess is that the small sample size tends to violate the i.i.d. assumption in some way, maybe because too many samples have observations closer to the mode. In the extreme case (N=1) the CLT would, of course, not even apply anymore. But what is the actual reason?

EDIT: To clarify, here is the central limit theorem from Wikipedia: Suppose $\{ X_1,...,X_n,... \}$ is a sequence of i.i.d. random variables with $E [X_i] = \mu$ and $Var [X_i] = \sigma^2 < \infty$. Then as $n$ approaches infinity, the random variables $\sqrt{n}(\overline{X}_n - \mu)$ converge in distribution to a normal $\mathcal{N} (0, \sigma^2)$: \begin{equation*} \sqrt{n}(\overline{X}_n - \mu) \stackrel{d}{\rightarrow} \mathcal{N} (0,\sigma^2). \end{equation*} If I draw $n$ samples of size $N$ from the population, the means calculated for each of these samples would follow a normal as $n \rightarrow \infty$.

Does the CLT refer to a large $n$ or a large $N$?

Answer 1

Let's state first what the CLT is about.

Let $X_1,\ldots,X_n$ be i.i.d. r.v.'s, with finite mean $\mu$ and finite variance $\sigma^2>0$ and let let $\bar{X}_n = \frac{1}{n}\sum_{j=1}^n X_j$.

$$ \frac{\sqrt{n}(\bar{X}_n - \mu)}{\sigma}\overset{d}\to \text{N}(0,1).\tag{*} $$

If we let $G_n = P\left[\frac{\sqrt{n}(\bar{X}_n-\mu)}{\sigma}\leq x\right]$ and $\Phi(x) = \int_{-\infty}^{x}\frac{1}{\sqrt{2\pi}}e^{-t^2/2}dt$, then (*) is equivalent to $$ \lim_{n\to\infty} G_n(x) = \Phi(x),\quad\text{for every }x\in \mathbb{R}. $$

The CLT says that for large $n$, the d.f. of the standardised sums $\frac{S_n-E(S_n)}{\sqrt{\text{var}(S_n)}}$ is close to $\Phi(x)$, where $S_n=\sum_{j=1}^n X_j$.

Thus, for every fixed $x$, we essentially have a convergent sequence $G_1(x), G_2(x),\ldots,$ with limit $\Phi(x)$. Thus, the larger $n$ the closer the sequence $G_n(x)$ to its limiting point $F(x)$.