Interpreting the Concept of 'Single Sample Normality' in the Context of the Central Limit Theorem

Question

In the context of the Central Limit Theorem (CLT), which postulates that the distribution of sample means will approximate a normal distribution given a sufficiently large number of samples and sample size, how can we reconcile this with the following concept presented by Rowntree in "Statistics Without Tears"? He claims that: "Even though we take only one sample, and therefore have only one sample means, we can think of it as belonging to a distribution of possible sample means. And, provided we are thinking of samples of reasonable size, this distribution will be normal"."

This concept seems to be in contrast with the binomial distribution resulting from a large single sample of a million coin flips, which does not resemble a normal distribution. But if we consider the means of multiple independent samples of coin flips, they do approximate a normal distribution as per the CLT.

So, how can we interpret Rowntree's claim?

Answer 1

It depends on how you are looking at the outcome of a million coin flips. If your register the outcomes 1 or 0, then sure, you get a distribution that concentrates on these two values. But these are then like a million samples of size 1 each. Otherwise, when you look at the mean of a million coin flips, and repeat this, you get results that are extremely close to 0.5 each time, but they still vary randomly.

In the CLT, we look at the distribution of the mean $\bar{X}$ from a sample of size $n$ that is multiplied with the $\sqrt{n}$. More precisely, we study the distribution of $Y = \sqrt{n}(\bar{X}-\mu)$, where $\mu$ is the theoretical mean of one outcome -- for a coin flip hopefully $\mu = 0.5$. The multiplication with factor $\sqrt{n}$ makes up for the fact, that the distribution of $\bar{X}$ concentrates more and more around $\mu$ when $n$ increases.

When you flip n=4 coins, say, you get possible outcomes for the sample mean 0, 0.25, 0.5, 0.75 and 1, so $Y= \sqrt{n}(\bar{X}-\mu)$ would have possible outcomes -1, -0.5, 0, 0.5 and 1. The probabilities for these outcomes are 1/16, 1/4, 3/8, 1/4, 1/16 respectively. You can graph this:

Doing the same with n=100 coin flips, the $\bar{X}$ can be 0, 0.01, .. 1, but $Y$ takes values $-5, -4.9, \dots, 5$.

Already for $n=4$, the shape of the distribution was somewhat bell shaped, but for $n=100$, it looks really normal.