I found the following definition of the Central Limit Theorem from book Probability and Statistics by Degroot (also from Wikipedia). It simply states the CLT as $$ \lim_{n\to\infty} \Pr\bigg[\frac{\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] = \Phi(z), \quad \cdots (1)$$ where $\mu$ is population mean, $\sigma$ is population standard deviation and $\bar{X}_n= \frac{1}{n}\sum_{i=i}^{n}X_i = \frac{1}{n} \left( X_1+X_2+ \cdots+X_n \right)$.

My concern is if $n \to \infty$ then why $\bar{X}$ would follow a nondegenerate distribution ? The law of large number states that $$ \Pr\!\left( \lim_{n\to\infty}\bar{X}_n = \mu \right) = 1$$ If this is true then both numerator and denominator in equation 1 will converge to zero as $n \to \infty$ and $\bar{X}$ would become a constant instead of following a specific distribution.

  • 1
    $\begingroup$ Several of the answers in our CLT thread at stats.stackexchange.com/questions/3734 address this question. $\endgroup$
    – whuber
    Feb 6 '16 at 16:09
  • 4
    $\begingroup$ The numerator and the denominator both converge to $0$ as $n \rightarrow \infty$. And we should recognize that $\frac{0}{0}$ is an indeterminate form. In general, the set of things giving us indeterminate forms can do whatever you like in the limit. This particular thing does something specific in the limit (as long as the hypotheses of the CLT are met, which you skip over in your post). $\endgroup$ Feb 6 '16 at 17:31
  • 3
    $\begingroup$ $\frac{\overline{X}_n-\mu}{\frac{\sigma}{\sqrt{n}}}$ is a sequence of random variables which all have mean zero and variance $1$. So it certainly can't converge to a degenerate distribution. The remarkable things about the CLT are that this sequence converges at all, and that the limit is universal among such a large class of $X_i$. $\endgroup$
    – Ian
    Feb 6 '16 at 20:23

Note that in your expression $$ \lim_{n\to\infty} \Pr\bigg[\frac{\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] $$ There is nowhere a reference to $\lim_{n\to\infty}\bar{X}_n$. It doesn't matter what this last part converges to - you're working with a different expression. It seems you were trying to do something like $$ \lim_{n\to\infty} \Pr\bigg[\frac{\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] = \Pr\bigg[\frac{\lim_{n\to\infty}\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] = \Pr\bigg[\frac{\mu-\mu}{\sigma/\sqrt n} \le z\bigg] = \mathrm{Pr}[0\le z]$$ But you can't do that, just like in normal calculus you can't do something like $$\lim_{t\to\infty}\left(\frac1t\cdot t\right)=\left(\lim_{t\to\infty}\frac1t\right)t=0.$$ In reality, it's true that as $n$ increases, the difference $\bar{X}_n-\mu$ becomes smaller, but you also multiply by $\sqrt{n}$ which gets larger and offsets this. The combined effect is that as $n$ increases, $\frac{\bar{X}_n-\mu}{\sigma/\sqrt n}$ becomes closer to a standard normal distribution.

  • $\begingroup$ +1. This is exactly what I was thinking. But at $n=\infty$, we would have $\bar{X}_n = \mu$. If this is true, then how difference would maintain even by multiplying $\sqrt{n}$ ? $\endgroup$
    – Neeraj
    Feb 10 '16 at 19:06
  • $\begingroup$ Can you give me some reference where I can find simple derivation of above proof ? $\endgroup$
    – Neeraj
    Feb 10 '16 at 19:08
  • 2
    $\begingroup$ @Neeraj: But at $n=\infty$ you're not looking at $\bar{X}_n$. You're looking at $(\bar{X}_n-\mu)/(\sigma/\sqrt{n})$. You first multiply by $\sqrt{n}$ and only then take the limit. You can't take the limit ahead of time. This is exactly the same in normal calculus - the order of operations matters. Look at the example I gave with t, and it's easy to come up with other examples. For example $\lim_{t\to0}((\sin t)/t) = 1$. Even though $\lim_{t\to0}\sin t=0$, the fact that you divide by $t$ before taking the limit matters. $\endgroup$ Feb 10 '16 at 21:31
  • 1
    $\begingroup$ @Neeraj: I'm not handy with references, but any relevant textbook should include a proof of the CLT, and it seems Wikipedia has one as well. $\endgroup$ Feb 10 '16 at 21:33
  • 1
    $\begingroup$ @Neeraj: Maybe it will help if you think of it less abstractly. Take a simple distribution for X, like coin toss or a die roll or whatever. For $n=1, 2, 3$, draw a graph of the distribution for $(\bar{X}_n-\mu)/(\sigma/\sqrt{n})$. You'll see how it becomes closer and closer to the standard normal. $\endgroup$ Feb 10 '16 at 21:35
  1. $\bar{X}$ is defined as $\frac{1}{n}\sum_{i=1}^nX_i$, your definition misses an average, and summation should start at $i=1$.
  2. $\mu$ is the population mean, not the sample mean ($\bar{X}$ is). Likewise, $\sigma$ is the population standard deviation.

  3. The crucial difference to the LLN is that the difference between $\bar{X}$ and $\mu$ (which indeed vanishes by the LLN) is scaled by $\sqrt{n}$, which diverges. So rewrite (1) as $\sqrt{n}(\bar{X}-\mu)/\sigma$, and it turns out (a proper proof would be too long here) that this product of two things, one of which tends to 0 and the other to infinity indeed (under suitable assumptions) still has a distribution asymptotically.

  • $\begingroup$ +1 Thanks. Can you please explain your 3rd point ? $\endgroup$
    – Neeraj
    Feb 6 '16 at 15:20
  • 1
    $\begingroup$ @Neeraj What do you mean? Take a few examples. Let X be an normally distributed RV. Then X/n goes to 0 as n goes to infinity, n itself goes to infinity as n goes to infinity, now if we take the product we have something that goes to 0 and something that goes to infinity as n gets larger and larger. Obviously though the n in the denominator and the n itself cancel and we're left with X which has a distribution. Basically something similar is happening when we do the scaling in step 3. I'm being very hand-wavy with this but hopefully it helps $\endgroup$
    – Dason
    Feb 6 '16 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.