4
$\begingroup$

What is wrong with the following sentence:

The Central Limit Theorem implies that, as the sample size grows, the error distribution approaches normality.

Am I correct by saying that it should in stead state the MEAN of the sample error approaches zero as sample size grows?

Improve this question
$\endgroup$
4
  • $\begingroup$ I meant to say "the distribution of the mean of the sample error approaches zero" Do you agree? Or am I missing the mistake in the statement completely? Thanks for the opinions (in advance) $\endgroup$
    – Las
    Commented Feb 9, 2012 at 16:07
  • 3
    $\begingroup$ You're going to need to be more specific (e.g. the error distribution of...? What model are you using?) but the second thing you said sound closer to the law of large numbers than the CLT. $\endgroup$
    – Macro
    Commented Feb 9, 2012 at 16:23
  • $\begingroup$ The error distribution does not change. $\endgroup$
    – Xi'an
    Commented Feb 9, 2012 at 19:00
  • 2
    $\begingroup$ It depends on what "error distribution" means, Xi'an (and what statistic it refers to, and what kind of asymptotic setting one is in, etc.). This question is impossible to answer without making (at least some implicit) assumptions about the context. It would help for the OP to respond to Macro's request for more information. $\endgroup$
    – whuber ♦
    Commented Feb 9, 2012 at 20:02

1 Answer 1

Reset to default
14
$\begingroup$

In its standard simplest form, the Central Limit Theorem (CLT) is a statement about the cumulative distribution function of the random variable $$Z_n = \frac{X_1 + X_2 + \cdots + X_n -n\mu}{\sigma \sqrt{n}}$$ where the $X_i$ are independent identically distributed random variables with mean $\mu$ and standard deviation $\sigma$. The CLT asserts that for each $a$, $-\infty < a < \infty$, $$F_{Z_n}(a) = P\left\{\frac{X_1 + X_2 + \cdots + X_n -n\mu}{\sigma \sqrt{n}} \leq a \right\} \to \Phi(a) = \int_{-\infty}^a \frac{e^{-x^2/2}}{\sqrt{2\pi}}\mathrm dx$$ as $n \to \infty$.

If by "error distribution" you mean the distribution function of $$Y_n = \left(\frac{1}{n}\sum_{i=1}^n X_i\right) -\mu = \frac{\sigma}{\sqrt{n}}Z_n,$$ that is, the difference of the sample mean $\bar{X} = n^{-1}\sum_iX_i$ and the population mean $\mu$, then the CLT certainly does not imply that $F_{Y_n}(\cdot)$ "approaches normality" as the sample size $n$ grows large in the usual sense of normality, though nitpickers may want to claim that the distribution is approaching a normal distribution with mean $0$ and standard deviation $0$ (often called a constant by statistically illiterate people).

On the other hand, the mean of the sample error is not a random variable but a constant (in fact, $0$ since the sample mean is an unbiased estimator of the population mean) and does not need to approach $0$; it is already there! I think what you meant to say is that the distribution $F_{Y_n}(a)$ of the sample error approaches the unit step function: $$F_{Y_n}(a) \to u(a) = \begin{cases}1, & \text{if}~a > 0,\\ 0 &\text{if}~a < 0,\end{cases}$$ which is certainly correct, and follows from the CLT, but also follows from results such as the weak law of large numbers which makes no assertions about the distribution of $Z_n$, only about $Y_n$.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ (+1) Here is a fun nitpick: One needs the extra assumption that $u(a)$ is continuous at $a$ to be able to conclude $F_{Y_n}(a) \to u(a)$, so, in particular, the first inequality ($a \geq 0$) also needs to be strict ($a > 0$), like the second one. A canonical example is to take $X_i$ iid uniform on $\{-1,+1\}$ and show that $F_{Y_n}(0) \to 1/2$. Showing this for the subsequence of odd $n$ is easiest (just symmetry), but it's not much harder to show it holds also for the subsequence of even $n$. $\endgroup$
    – cardinal
    Commented Feb 9, 2012 at 19:36
  • $\begingroup$ @cardinal You are right; in the interest of brevity, I had skipped stating the part where convergence is guaranteed only at points where $u(\cdot)$ is continuous, and then written the standard CDF description. Thanks for pointing out the mistake. I will fix the statement. $\endgroup$
    – Dilip Sarwate
    Commented Feb 9, 2012 at 19:48
  • $\begingroup$ This is all thew information I was given (old test question). But we focus on OLS models if that makes a difference at all. Thanks for the information. $\endgroup$
    – Las
    Commented Feb 10, 2012 at 15:39
  • $\begingroup$ @Las You and user9097 (cf. this question) seem to be looking at the same exam! If the question on the exam asks whether the statement "The CLT says the distribution of the sample mean asymptotically converges to a normal distribution" is correct, the answer is NO. If the exam goes on to ask "If you answered NO, what does the CLT say actually?", the answer is that subject to suitable conditions (which you can get from the above), the CLT says that the distribution of $\sqrt{n}\bar{X}$ converge to a normal $\mathcal N(\mu, \sigma)$ distribution. $\endgroup$
    – Dilip Sarwate
    Commented Feb 10, 2012 at 16:11
  • $\begingroup$ Thanks Dilip. That certainly makes more sense to me now. $\endgroup$
    – Las
    Commented Feb 10, 2012 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.