3
$\begingroup$

When you add two independent normal distributions the resulting distributions' variance is the sum of the variances i.e. it gets larger. However, the Central Limit Theorem states that

when independent random variables are added, their properly normalized sum tends toward a normal distribution (informally a bell curve) even if the original variables themselves are not normally distributed. https://en.wikipedia.org/wiki/Central_limit_theorem

and also says that as you add more variables the variance gets smaller.

Why does the variance increase in one case but decrease in the other? Is it because in the CLT case the random variables come from the same underlying distribution?

$\endgroup$
2
  • 4
    $\begingroup$ The CLT deals with sample mean $\bar X,$ for which the standard error is $SD(\bar X) = \sigma/\sqrt{n}.$ $\endgroup$
    – BruceET
    Nov 23, 2020 at 19:45
  • 1
    $\begingroup$ True, but how's this a comment and not an answer? $\endgroup$ Nov 23, 2020 at 21:16

2 Answers 2

3
$\begingroup$

Let $X_1, X_2\overset{iid}{\sim}F_X(x)$ with $\mu$ and $\sigma^2$ defined and finite.

$$var(X_1+X_2) = var(X_1) + var(X_2) = 2\sigma^2$$

But...

$$var(\bar{X}) = var\bigg(\dfrac{X_1 + X_2}{2}\bigg) = \dfrac{1}{4}var(X_1 + X_2) = \dfrac{1}{2}\sigma^2 $$

(Remember that variance is not linear; the constant pulls out of the variance operator as a squared value.)

Using more than two $iid$ variables results in that familiar $var(\bar{X}) = \dfrac{\sigma^2}{n}$.

$$var(\bar{X}) = var\bigg(\dfrac{1}{n}\sum_{i=1}^n X_i\bigg) = \dfrac{1}{n^2}var\bigg(\sum_{i=1}^n X_i\bigg) = \dfrac{1}{n^2} n\sigma^2 = \dfrac{\sigma^2}{n} $$

$\endgroup$
0
$\begingroup$

What I understand is that this question is asking why for

$X_1, X_2, ...\overset{iid}{\sim}F_X(x)$ with $X \sim N(\mu,\sigma^2)$

we have that

$Var(X_1), Var(X_1+X_2), Var(X_1+X_2+X_3), ...$ is increasing while

$Var\bigg(\dfrac{X_1}{1}\bigg), Var\bigg(\dfrac{X_1 + X_2}{2}\bigg), Var\bigg(\dfrac{X_1 + X_2 + X_3}{3}\bigg), ...$

is decreasing or at least non-increasing. I guess OP just overlooked that there's a fraction $\frac{\cdot}{n}$ multiplied to the sum.

Dave's answer explains why the 2nd sequence is decreasing, but actually even if variance were for example linear in taking out constants, the sequence is non-increasing due to being constant:

$(Var\bigg(\dfrac{X_1}{1}\bigg), Var\bigg(\dfrac{X_1 + X_2}{2}\bigg), Var\bigg(\dfrac{X_1 + X_2 + X_3}{3}\bigg), ...) = (\dfrac{1\sigma}{1}, \dfrac{2\sigma}{2}, \dfrac{3\sigma}{3}, ...)$

This post actually is not mainly to answer the question but to see if indeed this is what OP was overlooking: that there's a fraction $\frac{\cdot}{n}$ multiplied to the sum?

P.S. Re the wikipedia quote of OP the 'normalised' part links to this page which says

There are different types of normalizations in statistics – nondimensional ratios of errors, residuals, means and standard deviations, which are hence scale invariant – some of which may be summarized as follows.

So I guess the relevant part of the above quote is 'means' and thus the 'normalised sum' in OP's quote refers to taking the mean, i.e. refers to the missing fraction $\frac{\cdot}{n}$ multiplied to the sum.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.