# How can we get a normal distribution as $n \to \infty$ if the range of values of our random variable is bounded?

Let's say we have a random variable with a range of values bounded by $a$ and $b$, where $a$ is the minimum value and $b$ the maximum value.

I was told that as $n \to \infty$, where $n$ is our sample size, the sampling distribution of our sample means is a normal distribution. That is, as we increase $n$ we get closer and closer to a normal distribution, but the actual limit as $n \to \infty$ is equal to a normal distribution.

However, isn't part of the definition of the normal distribution that it has to extend from $- \infty$ to $\infty$?

If the max of our range is $b$, then the maximum sample mean (regardless of sample size) is going to be equal to $b$, and the minimum sample mean equal to $a$.

So it seems to me that even if we take the limit as $n$ approaches infinity, our distribution is not an actual normal distribution, because it is bounded by $a$ and $b$.

What am I missing?

Here is what you are missing. The asymptotic distribution is not of $\bar{X}_n$ (the sample mean), but of $\sqrt{n}(\bar{X}_n - \theta)$, where $\theta$ is the mean of $X$.

Let $X_1, X_2, \dots$ be iid random variables such that $a < X_i <b$ and $X_i$ has mean $\theta$ and variance $\sigma^2$. Thus $X_i$ has bounded support. The CLT says that $$\sqrt{n}(\bar{X}_n - \theta) \overset{d}{\to} N(0, \sigma^2),$$

where $\bar{X}_n$ is the sample mean. Now

\begin{align*} a < &X_i <b\\ a < & \bar{X}_n <b\\ a-\theta < &\bar{X}_n - \theta < b - \theta\\ \sqrt{n}(a - \theta) < & \sqrt{n}(\bar{X}_n - \theta) < \sqrt{n}(b - \theta).\\ \end{align*}

As $n \to \infty$, the lower bound and the upper bound tend to $-\infty$ and $\infty$ respectively, and thus as $n \to \infty$ the support of $\sqrt{n}(\bar{X}_n - \theta)$ is exactly the whole real line.

Whenever we use the CLT in practice, we say $\bar{X}_n \approx N(\theta, \sigma^2/n)$, and this will always be an approximation.

EDIT: I think part of the confusion is from the misinterpretation of the Central Limit Theorem. You are correct that the sampling distribution of the sample mean is $$\bar{X}_n \approx N(\theta, \sigma^2/n).$$

However, the sampling distribution is a finite sample property. Like you said, we want to let $n \to \infty$; once we do that the $\approx$ sign will be an exact result. However, if we let $n \to \infty$, we can no longer have an $n$ on the right hand side (since $n$ is now $\infty$). So the following statement is incorrect $$\bar{X}_n \overset{d}{\to} N(\theta, \sigma^2/n) \text{ as } n \to \infty.$$

[Here $\overset{d}{\to}$ stands for convergence in terms of distribution]. We want to write the result down accurately, so the $n$ is not on the right hand side. Here we now use properties of random variables to get

$$\sqrt{n}(\bar{X}_n - \theta) \overset{d}{\to} N(0, \sigma^2)$$

To see how the algebra works out, look at the answer here.

• Thank you. I understand your inequality algebra but I still have some confusion about your first paragraph: "The asymptotic distribution is not of $\bar{X}_n$ (the sample mean), but of $\sqrt{n} (\bar{X}_n - \theta)$...". I thought the CLT said that the sampling distribution of the sample means approaches a normal distribution as $n \to \infty$, and I thought $\bar{X}_n$ was the RV which takes on all possible values of samples of size $n$. Where does $\sqrt{n} (\bar{X}_n - \theta)$ come from? Why are we interested in that distribution and not the distribution of $\bar{X}_n$? – jeremy radcliff Jul 18 '16 at 4:05
• (cont'd) Is this about normalizing the distribution of the sample means? Is this where the square root comes from? Does it have to do with $Z$ scores? – jeremy radcliff Jul 18 '16 at 4:11
• @jeremyradcliff I have edited my answer, and included a link that explains some of the details. Hope this makes more sense now. – Greenparker Jul 18 '16 at 4:30
• Thank you so much for taking the time to edit, the link you provided is exactly what I was looking for. And you're right, the problem was that I had trouble reconciling the finite nature of the sampling distribution and the fact that we are taking $n$ to $\infty$. – jeremy radcliff Jul 18 '16 at 5:09

If you're referring to a central limit theorem, note that one proper way to to write it out is

$\left( \frac{\bar x - \mu} {\sigma} \right) \sqrt n \rightarrow_d N(0,1)$

under normal conditions ($\mu, \sigma$ being the mean and standard deviation of $x_i$).

With this formal definition, you can see right away that the left hand side can take on values for any finite range given a large enough $n$.

To help connect to the for informal idea that "a mean approaches a normal distribution for large $n$", we need to realize that "approaches a normal distribution" means that the CDF's get arbitrarily close to a normal distribution as $n$ gets large. But as $n$ gets large, the standard deviation of this approximate distribution shrinks, so the probability of an extreme tail of the approximating normal also goes to 0.

For example, suppose $X_i \sim \text{Bern}(p = 0.5)$. Then you could use the informal approximation to say that

$\bar X \dot \sim N\left(p, \frac{p(1-p)}{n}\right)$

So while it is true that for any finite $n$,

$P\left(N\left(p, \frac{p(1-p)}{n}\right) < 0\right) >0$

(implying the approximation is clearly never perfect), as $n \rightarrow \infty$,

$P\left(N\left(p, \frac{p(1-p)}{n}\right) < 0\right) \rightarrow 0$

So that discrepancy between the actual distribution and approximate distribution is disappearing, as is supposed to happen with approximations.