1
$\begingroup$

The Central Limit Theorem says that the distribution of the sample mean is approximately normal. Is there any intuitive explanation for why this should be so? I know it can be proven with deep math, but is there any explanation that makes it plausible why this distribution should be approximately normal?

I'm hoping for an explanation that doesn't require much mathematics, even if it is not a formal proof. I know how to explain what the mean and standard deviation of this distribution should be, but not why its shape is approximately normal.

$\endgroup$
3
  • $\begingroup$ Not sufficient for an answer, but you may find Galton boards visually motivating. $\endgroup$
    – Galen
    Oct 27 at 23:45
  • $\begingroup$ If there was intuition it wouldn't take someone like de Moivre to first postulate it. Nothing that's not mathematical could lead to something like this. It's not like elders in my grandpa's village would have a proverb for CLT. Bad luck on this, my non-mathematician friend $\endgroup$
    – Aksakal
    Oct 27 at 23:51
  • 1
    $\begingroup$ Thank you @Galen! $\endgroup$
    – D.W.
    Oct 28 at 4:49
2
$\begingroup$

What if we start with a simpler example? The best place is a binary distribution.

CLT for Binary Variables

Suppose that $X$ takes two values $\{\mu-\sigma,\mu+\sigma)$. Think of this as a coin toss between a high and a low value: $$ \mathbb{P}(X = \mu - \sigma) = \frac{1}{2} $$ $$ \mathbb{P}(X = \mu + \sigma) = \frac{1}{2} $$ It is straightfoward to check that $X$ has mean and variance $(\mu,\sigma^2)$. The simplest version of the central limit theorem says that if $\{X_1,\ldots,X_n\}$ are i.i.d. and $\sigma^2 < \infty$, then the sample average $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$ satisfies the following property as $n \to \infty$ $$ \frac{\sqrt{N}(\bar{X} - \mu)}{\sigma} \to^d \mathcal{N}(0,1) $$

An intuitive explanation

Why don't we break down the left-hand side in our example? $$ \frac{\sqrt{N}(\bar{X} - \mu)}{\sigma} = \frac{1}{\sqrt{n}} \sum_{i=1}^n \frac{X_i - \mu}{\sigma} $$ Then the normalized variable $Z_i = \frac{X_i - \mu}{\sigma}$ is actually a binary variable that takes on values $\{-1,1\}$ with probability $\frac{1}{2}$. Then it follows that $$ \sum_{i=1}^n Z_i\sim 2 * Binomial(n,p=0.5) - n$$ The binomial already has a bell-shaped curve. This comes from the fact that observing a series of extreme events with high draws of $Z_i$ is very unlikely (like in a coin toss). As $n$ gets larger, the sum actually approximates the normal distribution. Dividing by $\sqrt{n}$ is important so that the distribution doesn't diverge to infinity.

Main takeaways for general case

  • The central limit theorem is very related to combinations of independent events.
  • Is it likely to observe a series of extreme events if observations are i.i.d.? It isn't.
    • The general formulation extends this logic.
    • Even if $X$ has a continuous distribution, it is still very unlikely to see a sequence of high values.
  • Intuitively $\ldots$ counting sequences of events above and below the mean turns out to be more important than the actual values they take. To take into account the dispersion in values you just need to normalize $\bar{X}$ by the variance.
    • Hence the most important assumption is that the events are independent (and that $\sigma^2$ is finite).
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.