Asymptotic normality in central limit theorem

Question

I am a bit confused by Classical CLT section of the central limit theorem on Wikipedia. It basically says at the sample size gets larger, the difference between the sample mean and true mean approximates the normal distribution. But from the law of large numbers we know that as sample size approaches infinity, the sample mean becomes the true mean. In other words, it is a single point instead of a distribution. So this result cannot be talking about the limiting case. So at which point can we regard the difference between the sample mean and true mean to be normal?

Answer 1

Both types of convergence hold

I think that the distinction you are referring to gets somewhat lost in a formal explanation, so allow me to give you a more informal heuristic view. Re-arranging the formal convergence in distribution result in the CLT result gives the following asymptotic approximating distribution:

$$\bar{X}_n \overset{\text{Approx}}{\sim} \text{N} \bigg( \mu, \frac{\sigma^2}{n} \bigg) \quad \quad \quad \text{for large } n.$$

However, as you can see, as $n \rightarrow \infty$ the variance in this approximating distribution approaches zero and so $\bar{X}_n \rightarrow \mu$ in probability. So both of the results you raise are true --- the distribution of the sample mean converges to the normal distribution, and the variance of that distribution converges to zero, such that the sample mean converges (in probability) to the true mean.

Answer 2

Your confusion is perhaps due to the fact that there are different modes of convergence of a sequence of random variables. Definitions first. The CLT tells us that:

under suitable conditions, the standardized sample average has limiting distribution $N(0,1)$

that is

$$ \frac{\sqrt{n}(\bar X_n -\mu)}{\sigma}\overset{d}\to N(0,1). $$

On the other hand, the (Weak) Law of Large Numbers (LLN) deals with averages and tells us that

Under suitable conditions, the distribution of the sample average is the point mass at $\mu$ (the population average)

that is

$\bar X \overset{P}\to \mu.$

The message here is that the CLT is an instance of convergence in distribution whereas the LLN is an instance of convergence in probability. The fact that $\bar X \overset{P}\to \mu$, i.e. that $\bar X-\mu \overset{P}\to 0$ doesn't conflict with CLT because LLN targets the (centred) sample average whereas the CLT targets the standardized sample average.