12
$\begingroup$

Let's say we have a random variable with a range of values bounded by $a$ and $b$, where $a$ is the minimum value and $b$ the maximum value.

I was told that as $n \to \infty$, where $n$ is our sample size, the sampling distribution of our sample means is a normal distribution. That is, as we increase $n$ we get closer and closer to a normal distribution, but the actual limit as $n \to \infty$ is equal to a normal distribution.

However, isn't part of the definition of the normal distribution that it has to extend from $- \infty$ to $\infty$?

If the max of our range is $b$, then the maximum sample mean (regardless of sample size) is going to be equal to $b$, and the minimum sample mean equal to $a$.

So it seems to me that even if we take the limit as $n$ approaches infinity, our distribution is not an actual normal distribution, because it is bounded by $a$ and $b$.

What am I missing?

$\endgroup$
15
$\begingroup$

Here is what you are missing. The asymptotic distribution is not of $\bar{X}_n$ (the sample mean), but of $\sqrt{n}(\bar{X}_n - \theta)$, where $\theta$ is the mean of $X$.

Let $X_1, X_2, \dots$ be iid random variables such that $a < X_i <b$ and $X_i$ has mean $\theta$ and variance $\sigma^2$. Thus $X_i$ has bounded support. The CLT says that $$\sqrt{n}(\bar{X}_n - \theta) \overset{d}{\to} N(0, \sigma^2), $$

where $\bar{X}_n$ is the sample mean. Now

\begin{align*} a < &X_i <b\\ a < & \bar{X}_n <b\\ a-\theta < &\bar{X}_n - \theta < b - \theta\\ \sqrt{n}(a - \theta) < & \sqrt{n}(\bar{X}_n - \theta) < \sqrt{n}(b - \theta).\\ \end{align*}

As $n \to \infty$, the lower bound and the upper bound tend to $-\infty$ and $\infty$ respectively, and thus as $n \to \infty$ the support of $\sqrt{n}(\bar{X}_n - \theta)$ is exactly the whole real line.

Whenever we use the CLT in practice, we say $\bar{X}_n \approx N(\theta, \sigma^2/n)$, and this will always be an approximation.


EDIT: I think part of the confusion is from the misinterpretation of the Central Limit Theorem. You are correct that the sampling distribution of the sample mean is $$\bar{X}_n \approx N(\theta, \sigma^2/n). $$

However, the sampling distribution is a finite sample property. Like you said, we want to let $n \to \infty$; once we do that the $\approx$ sign will be an exact result. However, if we let $n \to \infty$, we can no longer have an $n$ on the right hand side (since $n$ is now $\infty$). So the following statement is incorrect $$ \bar{X}_n \overset{d}{\to} N(\theta, \sigma^2/n) \text{ as } n \to \infty.$$

[Here $\overset{d}{\to}$ stands for convergence in terms of distribution]. We want to write the result down accurately, so the $n$ is not on the right hand side. Here we now use properties of random variables to get

$$ \sqrt{n}(\bar{X}_n - \theta) \overset{d}{\to} N(0, \sigma^2)$$

To see how the algebra works out, look at the answer here.

$\endgroup$
  • $\begingroup$ Thank you. I understand your inequality algebra but I still have some confusion about your first paragraph: "The asymptotic distribution is not of $\bar{X}_n$ (the sample mean), but of $\sqrt{n} (\bar{X}_n - \theta)$...". I thought the CLT said that the sampling distribution of the sample means approaches a normal distribution as $n \to \infty$, and I thought $\bar{X}_n$ was the RV which takes on all possible values of samples of size $n$. Where does $\sqrt{n} (\bar{X}_n - \theta)$ come from? Why are we interested in that distribution and not the distribution of $\bar{X}_n$? $\endgroup$ – jeremy radcliff Jul 18 '16 at 4:05
  • $\begingroup$ (cont'd) Is this about normalizing the distribution of the sample means? Is this where the square root comes from? Does it have to do with $Z$ scores? $\endgroup$ – jeremy radcliff Jul 18 '16 at 4:11
  • $\begingroup$ @jeremyradcliff I have edited my answer, and included a link that explains some of the details. Hope this makes more sense now. $\endgroup$ – Greenparker Jul 18 '16 at 4:30
  • 1
    $\begingroup$ Thank you so much for taking the time to edit, the link you provided is exactly what I was looking for. And you're right, the problem was that I had trouble reconciling the finite nature of the sampling distribution and the fact that we are taking $n$ to $\infty$. $\endgroup$ – jeremy radcliff Jul 18 '16 at 5:09
7
$\begingroup$

If you're referring to a central limit theorem, note that one proper way to to write it out is

$\left( \frac{\bar x - \mu} {\sigma} \right) \sqrt n \rightarrow_d N(0,1)$

under normal conditions ($\mu, \sigma$ being the mean and standard deviation of $x_i$).

With this formal definition, you can see right away that the left hand side can take on values for any finite range given a large enough $n$.

To help connect to the for informal idea that "a mean approaches a normal distribution for large $n$", we need to realize that "approaches a normal distribution" means that the CDF's get arbitrarily close to a normal distribution as $n$ gets large. But as $n$ gets large, the standard deviation of this approximate distribution shrinks, so the probability of an extreme tail of the approximating normal also goes to 0.

For example, suppose $X_i \sim \text{Bern}(p = 0.5)$. Then you could use the informal approximation to say that

$\bar X \dot \sim N\left(p, \frac{p(1-p)}{n}\right)$

So while it is true that for any finite $n$,

$P\left(N\left(p, \frac{p(1-p)}{n}\right) < 0\right) >0$

(implying the approximation is clearly never perfect), as $n \rightarrow \infty$,

$P\left(N\left(p, \frac{p(1-p)}{n}\right) < 0\right) \rightarrow 0$

So that discrepancy between the actual distribution and approximate distribution is disappearing, as is supposed to happen with approximations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.