3
$\begingroup$

I am a bit confused by Classical CLT section of the central limit theorem on Wikipedia. It basically says at the sample size gets larger, the difference between the sample mean and true mean approximates the normal distribution. But from the law of large numbers we know that as sample size approaches infinity, the sample mean becomes the true mean. In other words, it is a single point instead of a distribution. So this result cannot be talking about the limiting case. So at which point can we regard the difference between the sample mean and true mean to be normal?

$\endgroup$
4
  • 2
    $\begingroup$ check this stats.stackexchange.com/a/599253/56940 and the other links provided there. $\endgroup$
    – utobi
    Apr 18, 2023 at 6:48
  • 5
    $\begingroup$ You should account for the $\sqrt n$ in front... $\endgroup$
    – Xi'an
    Apr 18, 2023 at 9:27
  • 2
    $\begingroup$ The CLT says absolutely nothing about how large $n$ needs to be for it to be appropriate to view the sampling distribution of the mean as Normal. The Barry-Esseen theorem does have something (theoretical) to say about the matter: it tells you the skewness (standardized third moment) of the underlying distribution gives useful information about the rate of convergence to normality. $\endgroup$
    – whuber
    Apr 18, 2023 at 13:03
  • $\begingroup$ "the difference between the sample mean and true mean approximates the normal distribution. " - almost, the scaled difference (magnify by $\sqrt{n}$) approaches normal. The SLLN says that the unscaled difference goes to zero. When the mean exists! $\endgroup$
    – AdamO
    Apr 18, 2023 at 21:13

2 Answers 2

2
$\begingroup$

Both types of convergence hold

I think that the distinction you are referring to gets somewhat lost in a formal explanation, so allow me to give you a more informal heuristic view. Re-arranging the formal convergence in distribution result in the CLT result gives the following asymptotic approximating distribution:

$$\bar{X}_n \overset{\text{Approx}}{\sim} \text{N} \bigg( \mu, \frac{\sigma^2}{n} \bigg) \quad \quad \quad \text{for large } n.$$

However, as you can see, as $n \rightarrow \infty$ the variance in this approximating distribution approaches zero and so $\bar{X}_n \rightarrow \mu$ in probability. So both of the results you raise are true --- the distribution of the sample mean converges to the normal distribution, and the variance of that distribution converges to zero, such that the sample mean converges (in probability) to the true mean.

$\endgroup$
1
$\begingroup$

Your confusion is perhaps due to the fact that there are different modes of convergence of a sequence of random variables. Definitions first. The CLT tells us that:

under suitable conditions, the standardized sample average has limiting distribution $N(0,1)$

that is

$$ \frac{\sqrt{n}(\bar X_n -\mu)}{\sigma}\overset{d}\to N(0,1). $$

On the other hand, the (Weak) Law of Large Numbers (LLN) deals with averages and tells us that

Under suitable conditions, the distribution of the sample average is the point mass at $\mu$ (the population average)

that is

$\bar X \overset{P}\to \mu.$

The message here is that the CLT is an instance of convergence in distribution whereas the LLN is an instance of convergence in probability. The fact that $\bar X \overset{P}\to \mu$, i.e. that $\bar X-\mu \overset{P}\to 0$ doesn't conflict with CLT because LLN targets the (centred) sample average whereas the CLT targets the standardized sample average.

$\endgroup$
1
  • $\begingroup$ I see your post as "confusion about CLT vs LLN", so from a different perspective wrt whuber (check his comment). Feel free to ask for clarification if needed. $\endgroup$
    – utobi
    Apr 18, 2023 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.