10
$\begingroup$

Let $X_1,...,X_n$ be independent observations from a distribution that has the mean $\mu$ and variance $\sigma^2 < \infty$, when $n \rightarrow \infty$, then

$$\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma} \rightarrow N(0,1).$$

Why does this imply that $$\bar{X}_n \sim N\left(\mu, \frac{\sigma^2}{n}\right)?$$

$\endgroup$
  • $\begingroup$ Maybe this was not stressed clearly enough below, but the statement $$\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma} \rightarrow N(0,1)$$ is mathematically meaningful and true while the statement $$\bar{X}_n \sim N\left(\mu, \frac{\sigma^2}{n}\right)$$ is mathematically absurd, hence, as the saying goes, not even wrong. $\endgroup$ – Did Jan 20 at 15:19
17
$\begingroup$

You interpretation is slightly incorrect. The Central Limit Theorem (CLT) implies that

$$\bar{X}_n \overset{\mbox{approx}}{\sim} N \left(\mu, \frac{\sigma^2}{n} \right). $$

This is because CLT is an asymptotic result, and we are in practice dealing with only finite samples. However, when the sample size is large enough, then we assume that the CLT result holds true in approximation, and thus

\begin{align*} \sqrt{n} \dfrac{\bar{X}_n - \mu}{\sigma} &\overset{\mbox{approx}}{\sim} N(0, 1)\\ \sqrt{n} \dfrac{\bar{X}_n - \mu}{\sigma} . \dfrac{\sigma}{\sqrt{n}} &\overset{\mbox{approx}}{\sim} \dfrac{\sigma}{\sqrt{n}} N \left( 0, 1 \right)\\ {\bar{X}_n - \mu} &\overset{\mbox{approx}}{\sim} N \left(0, \dfrac{\sigma^2}{n}\right)\\ \bar{X}_n - \mu + \mu & \overset{\mbox{approx}}{\sim}\mu + N \left(0, \frac{\sigma^2}{n} \right)\\ \bar{X}_n & \overset{\mbox{approx}}{\sim} N \left(\mu, \frac{\sigma^2}{n} \right).\\ \end{align*}

This is because for a random variable $X$ and constants $a,b $, $\operatorname{Var}(aX) = a^2 \operatorname{Var}(X)$ (this is used in the second step) and $E(b + X) = b + E(X)$, $\operatorname{Var}(b + X) = \operatorname{Var}(X)$ (this is used in the second last step).

Read this for more explanation of the algebra.

$\endgroup$
  • $\begingroup$ Could you clarify what "algebra" are you using when taking the terms from the L.H.S. of $\sim$ to the R.H.S? $\endgroup$ – mavavilj Mar 24 '16 at 13:38
  • $\begingroup$ I have clarified the algebra. Most of it is using properties of variance and expectation. $\endgroup$ – Greenparker Mar 24 '16 at 13:45
  • $\begingroup$ Why does not e.g. the second term of $N(\mu, \frac{\sigma^2}{n})$ become $N(\mu, \mu+\frac{\sigma^2}{n})$? $\endgroup$ – mavavilj Mar 24 '16 at 13:51
  • 3
    $\begingroup$ Because the $Var(aX + b) = a^2 Var(X)$. Intuitively, adding a constant number to a random variable does not change its variance. $\endgroup$ – Greenparker Mar 24 '16 at 13:59
10
$\begingroup$

The easiest way to see this is by looking at the mean and the variance of the random variable $\bar X_n$.

So, $\mathcal{N}(0,1)$ states that the mean is zero and the variance is one. Hence, we have for the mean:

$$E\left[\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma}\right]\approx 0$$ Using $E[a\cdot x+b]=a\cdot E[x]+b$, where $a,b$ are constants, we get: $$\bar{X}_n\approx\mu$$

Now, using $\operatorname{Var}[a\cdot x+b]=a^2\cdot \operatorname{Var}[x]=a^2\cdot \sigma_x^2$, where $a,b$ are constants, we get the following for the variance:

$$\operatorname{Var}\left[\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma}\right]\approx 1$$ $$\operatorname{Var}\left[\bar{X}_n\right]\approx \frac{\sigma^2}{n}$$

Now, we know the mean and the variance of $\bar X_n$, and the Gaussian (normal) distribution with these mean and variance is $\mathcal{N}(\mu,\frac{\sigma^2}{n})$

You may wonder why go through all these algebra? Why not directly prove that $\bar X_n$ converges to $\mathcal{N}(\mu,\frac{\sigma^2}{n})$?

The reason is that in mathematics it's difficult (impossible?) to prove convergence to changing things, i.e. the right had side of the convergence operator $\rightarrow$ has to be fixed in order for mathematicians to use their tricks for proving statements. The $\mathcal{N}(\mu,\frac{\sigma^2}{n})$ expression changes with $n$, which is a problem. So, mathematicians transform the expressions in such a way, that the right hand side is fixed, e.g. $\mathcal{N}(0,1)$ is a nice fixed right hand side.

$\endgroup$
4
$\begingroup$

It doesn't imply the normality of $\bar{X}_n$, except as an approximation. But if we pretend for a moment that $ \sqrt{n} (\bar{X}_n - \mu) / \sigma$ is exactly standard normal then we have the result that $\tau Z + \mu \sim$ normal$(\mu, \tau^2)$ when $Z \sim$ normal$(0, 1)$. One way to see this is via the moment generating function

\begin{align} M_{\tau Z + \mu}(t) &= M_Z(\tau t) M_\mu (t) \\ &= e^{t^2 \tau^2 / 2} e^{t \mu} \\ &= e^{t^2 \tau^2 / 2 + t \mu} \end{align}

which is the normal$(\mu, \tau^2)$ m.g.f.

$\endgroup$
  • $\begingroup$ Why does the moment generating function prove it for the distribution? $\endgroup$ – mavavilj Mar 24 '16 at 13:42
  • 1
    $\begingroup$ This is a result from probability. If two random variables have the same moment generating function then they are equal in distribution. $\endgroup$ – dsaxton Mar 24 '16 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.