# Why does the variance decrease when averaging zero-mean gaussians?

I have the following in my textbook:

$$r_k \thicksim N(0,\sigma_r^2) \\ \Rightarrow \sum\limits_{k=1}^K r_k(t) \thicksim N(0,K\sigma_r^2) \\ \Rightarrow \frac1{K} \sum\limits_{k=1}^K r_k(t) \thicksim N(0,\frac{K}{K^2}\sigma_r^2) = N(0,\frac1{K}\sigma_r^2)$$

The context is averaging noise away. I have a feeling that this is correct but I can't see it mathematically:
Question 1:
How do I get from here: $$\sum\limits_{k=1}^K N(0,\sigma_r^2) = \sum\limits_{k=1}^K \frac1{\sqrt{2\pi \sigma_r^2}}e^{-\frac{(x-0)^2}{2\sigma_r^2}}$$ to here: $$N(0,K\sigma_r^2) = \frac1{\sqrt{2\pi K\sigma_r^2}}e^{-\frac{(x-0)^2}{2K\sigma_r^2}}$$
solved by aussetg, many thanks. I was interpreting $\sim$ as a fancy $=$. I learned that this is wrong: The sum of 2 dices is not twice the sum of one dice, compare probability of getting a seven.
Knowing that given
$X \thicksim N(\mu_X, \sigma_X^2), Y \thicksim N(\mu_Y, \sigma_Y^2)$
$Z(=X+Y) \thicksim N(\mu_X + \mu_Y, \sigma_X^2+\sigma_Y^2)$ It becomes clear that $\sum\limits_{k=1}^K r_k(t) \thicksim N(0,K\sigma_r^2)$

Question 2 how to get from here: $$\frac1{K} N(0,K\sigma_r^2) =\frac1{K} \frac1{\sqrt{2\pi K\sigma_r^2}}e^{-\frac{(x-0)^2}{2K\sigma_r^2}}$$ to here: $$N(0,\frac1{K}\sigma_r^2) = \frac1{\sqrt{2\pi \frac1{K}\sigma_r^2}}e^{-\frac{(x-0)^2}{2\frac1{K}\sigma_r^2}}$$

This is really easy once you realise $\sim$ != $=$.
If $var(X) = \sum\limits_i (\bar{x} - x_i)^2$ then $var(aX) = \sum\limits_i (a\bar{x} - ax_i)^2 = a^2\sum\limits_i (\bar{x} - x_i)^2 = a^2var(x)$
Therefore if $r'\sim N(0,K\sigma_r^2) \Rightarrow \frac1{K}r'\sim N(0,(\frac1{K})^2K\sigma_r^2)$

• You need to take an introductory probability class. Jul 6 '15 at 13:55

Don't sum densities, it doesn't mean anything. In particular, the density of a sum of random variables is not the sum of the densities. You may want to consider the case of rolling one vs. two dice to gain intuition here. Here's a reference.

What you need to do is prove by induction that the sum of independant gaussians is a gaussian.

The reduction of variance part in itself is trivially obtained by the definition

$$\mathbb{V}(X) = \mathbb{E} [ (X-\mathbb{E}(X))^2 ]$$

by observing it's a quadratic form. ( the covariance is actually a bilinear symmetric function ). Just plug your sum and use the independance of the variables, you will find that the sum contribute only to a factor $k$ while the leading $1/k$ contribute to $1/k^2$.

• I think it would improve your answer to 1) define quadratic form, 2) explicitly point out that the independence assumption makes the variance additive. Jul 6 '15 at 12:29

This is a general property of averaging processes and not specific to the normal distribution. If you have independent and identically distributed random variables $X_1, X_2, \ldots , X_n$ with $\text{Var}(X_1) = \sigma^2 < \infty$ then,

\begin{align} \text{Var} \left ( \frac{\sum_{i=1}^{n} X_i}{n} \right ) &= \frac{1}{n^2} \sum_{i=1}^{n} \text{Var}(X_i) \\ &= \frac{n \sigma^2}{n^2} \\ &= \frac{\sigma^2}{n} \end{align}

This makes sense if we think of each observation as a measurement of some average value $\mu$ plus an error term: $X_i = \mu + \epsilon_i$, where $\text{E}(\epsilon_i) = 0$. If we average up enough of these, we expect the error terms to roughly cancel and for the average to become increasingly concentrated around $\mu$.