5
$\begingroup$

I found the following definition of the Central Limit Theorem from book Probability and Statistics by Degroot (also from Wikipedia). It simply states the CLT as $$ \lim_{n\to\infty} \Pr\bigg[\frac{\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] = \Phi(z), \quad \cdots (1)$$ where $\mu$ is population mean, $\sigma$ is population standard deviation and $\bar{X}_n= \frac{1}{n}\sum_{i=i}^{n}X_i = \frac{1}{n} \left( X_1+X_2+ \cdots+X_n \right)$.

My concern is if $n \to \infty$ then why $\bar{X}$ would follow a nondegenerate distribution ? The law of large number states that $$ \Pr\!\left( \lim_{n\to\infty}\bar{X}_n = \mu \right) = 1$$ If this is true then both numerator and denominator in equation 1 will converge to zero as $n \to \infty$ and $\bar{X}$ would become a constant instead of following a specific distribution.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Several of the answers in our CLT thread at stats.stackexchange.com/questions/3734 address this question. $\endgroup$
    – whuber ♦
    Feb 6, 2016 at 16:09
  • 4
    $\begingroup$ The numerator and the denominator both converge to $0$ as $n \rightarrow \infty$. And we should recognize that $\frac{0}{0}$ is an indeterminate form. In general, the set of things giving us indeterminate forms can do whatever you like in the limit. This particular thing does something specific in the limit (as long as the hypotheses of the CLT are met, which you skip over in your post). $\endgroup$
    – Eric Towers
    Feb 6, 2016 at 17:31
  • 3
    $\begingroup$ $\frac{\overline{X}_n-\mu}{\frac{\sigma}{\sqrt{n}}}$ is a sequence of random variables which all have mean zero and variance $1$. So it certainly can't converge to a degenerate distribution. The remarkable things about the CLT are that this sequence converges at all, and that the limit is universal among such a large class of $X_i$. $\endgroup$
    – Ian
    Feb 6, 2016 at 20:23

2 Answers 2

Reset to default
7
$\begingroup$

Note that in your expression $$ \lim_{n\to\infty} \Pr\bigg[\frac{\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] $$ There is nowhere a reference to $\lim_{n\to\infty}\bar{X}_n$. It doesn't matter what this last part converges to - you're working with a different expression. It seems you were trying to do something like $$ \lim_{n\to\infty} \Pr\bigg[\frac{\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] = \Pr\bigg[\frac{\lim_{n\to\infty}\bar{X}_n-\mu}{\sigma/\sqrt n} \le z\bigg] = \Pr\bigg[\frac{\mu-\mu}{\sigma/\sqrt n} \le z\bigg] = \mathrm{Pr}[0\le z]$$ But you can't do that, just like in normal calculus you can't do something like $$\lim_{t\to\infty}\left(\frac1t\cdot t\right)=\left(\lim_{t\to\infty}\frac1t\right)t=0.$$ In reality, it's true that as $n$ increases, the difference $\bar{X}_n-\mu$ becomes smaller, but you also multiply by $\sqrt{n}$ which gets larger and offsets this. The combined effect is that as $n$ increases, $\frac{\bar{X}_n-\mu}{\sigma/\sqrt n}$ becomes closer to a standard normal distribution.

Improve this answer
$\endgroup$
5
  • $\begingroup$ +1. This is exactly what I was thinking. But at $n=\infty$, we would have $\bar{X}_n = \mu$. If this is true, then how difference would maintain even by multiplying $\sqrt{n}$ ? $\endgroup$
    – Neeraj
    Feb 10, 2016 at 19:06
  • $\begingroup$ Can you give me some reference where I can find simple derivation of above proof ? $\endgroup$
    – Neeraj
    Feb 10, 2016 at 19:08
  • 2
    $\begingroup$ @Neeraj: But at $n=\infty$ you're not looking at $\bar{X}_n$. You're looking at $(\bar{X}_n-\mu)/(\sigma/\sqrt{n})$. You first multiply by $\sqrt{n}$ and only then take the limit. You can't take the limit ahead of time. This is exactly the same in normal calculus - the order of operations matters. Look at the example I gave with t, and it's easy to come up with other examples. For example $\lim_{t\to0}((\sin t)/t) = 1$. Even though $\lim_{t\to0}\sin t=0$, the fact that you divide by $t$ before taking the limit matters. $\endgroup$
    – Meni Rosenfeld
    Feb 10, 2016 at 21:31
  • 1
    $\begingroup$ @Neeraj: I'm not handy with references, but any relevant textbook should include a proof of the CLT, and it seems Wikipedia has one as well. $\endgroup$
    – Meni Rosenfeld
    Feb 10, 2016 at 21:33
  • 1
    $\begingroup$ @Neeraj: Maybe it will help if you think of it less abstractly. Take a simple distribution for X, like coin toss or a die roll or whatever. For $n=1, 2, 3$, draw a graph of the distribution for $(\bar{X}_n-\mu)/(\sigma/\sqrt{n})$. You'll see how it becomes closer and closer to the standard normal. $\endgroup$
    – Meni Rosenfeld
    Feb 10, 2016 at 21:35
7
$\begingroup$
  1. $\bar{X}$ is defined as $\frac{1}{n}\sum_{i=1}^nX_i$, your definition misses an average, and summation should start at $i=1$.

  2. $\mu$ is the population mean, not the sample mean ($\bar{X}$ is). Likewise, $\sigma$ is the population standard deviation.

  3. The crucial difference to the LLN is that the difference between $\bar{X}$ and $\mu$ (which indeed vanishes by the LLN) is scaled by $\sqrt{n}$, which diverges. So rewrite (1) as $\sqrt{n}(\bar{X}-\mu)/\sigma$, and it turns out (a proper proof would be too long here) that this product of two things, one of which tends to 0 and the other to infinity indeed (under suitable assumptions) still has a distribution asymptotically.

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1 Thanks. Can you please explain your 3rd point ? $\endgroup$
    – Neeraj
    Feb 6, 2016 at 15:20
  • 1
    $\begingroup$ @Neeraj What do you mean? Take a few examples. Let X be an normally distributed RV. Then X/n goes to 0 as n goes to infinity, n itself goes to infinity as n goes to infinity, now if we take the product we have something that goes to 0 and something that goes to infinity as n gets larger and larger. Obviously though the n in the denominator and the n itself cancel and we're left with X which has a distribution. Basically something similar is happening when we do the scaling in step 3. I'm being very hand-wavy with this but hopefully it helps $\endgroup$
    – Dason
    Feb 6, 2016 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.