I have the following in my textbook:

$$ r_k \thicksim N(0,\sigma_r^2) \\ \Rightarrow \sum\limits_{k=1}^K r_k(t) \thicksim N(0,K\sigma_r^2) \\ \Rightarrow \frac1{K} \sum\limits_{k=1}^K r_k(t) \thicksim N(0,\frac{K}{K^2}\sigma_r^2) = N(0,\frac1{K}\sigma_r^2) $$

The context is averaging noise away. I have a feeling that this is correct but I can't see it mathematically:
Question 1:
How do I get from here: $$\sum\limits_{k=1}^K N(0,\sigma_r^2) = \sum\limits_{k=1}^K \frac1{\sqrt{2\pi \sigma_r^2}}e^{-\frac{(x-0)^2}{2\sigma_r^2}}$$ to here: $$N(0,K\sigma_r^2) = \frac1{\sqrt{2\pi K\sigma_r^2}}e^{-\frac{(x-0)^2}{2K\sigma_r^2}}$$
solved by aussetg, many thanks. I was interpreting $\sim$ as a fancy $=$. I learned that this is wrong: The sum of 2 dices is not twice the sum of one dice, compare probability of getting a seven.
Knowing that given
$X \thicksim N(\mu_X, \sigma_X^2), Y \thicksim N(\mu_Y, \sigma_Y^2)$
$Z(=X+Y) \thicksim N(\mu_X + \mu_Y, \sigma_X^2+\sigma_Y^2)$ It becomes clear that $\sum\limits_{k=1}^K r_k(t) \thicksim N(0,K\sigma_r^2)$

Question 2 how to get from here: $$\frac1{K} N(0,K\sigma_r^2) =\frac1{K} \frac1{\sqrt{2\pi K\sigma_r^2}}e^{-\frac{(x-0)^2}{2K\sigma_r^2}}$$ to here: $$N(0,\frac1{K}\sigma_r^2) = \frac1{\sqrt{2\pi \frac1{K}\sigma_r^2}}e^{-\frac{(x-0)^2}{2\frac1{K}\sigma_r^2}}$$

This is really easy once you realise $\sim$ != $=$.
If $var(X) = \sum\limits_i (\bar{x} - x_i)^2$ then $var(aX) = \sum\limits_i (a\bar{x} - ax_i)^2 = a^2\sum\limits_i (\bar{x} - x_i)^2 = a^2var(x)$
Therefore if $r'\sim N(0,K\sigma_r^2) \Rightarrow \frac1{K}r'\sim N(0,(\frac1{K})^2K\sigma_r^2)$

  • $\begingroup$ You need to take an introductory probability class. $\endgroup$ Jul 6 '15 at 13:55

Don't sum densities, it doesn't mean anything. In particular, the density of a sum of random variables is not the sum of the densities. You may want to consider the case of rolling one vs. two dice to gain intuition here. Here's a reference.

What you need to do is prove by induction that the sum of independant gaussians is a gaussian.

The reduction of variance part in itself is trivially obtained by the definition

$$ \mathbb{V}(X) = \mathbb{E} [ (X-\mathbb{E}(X))^2 ] $$

by observing it's a quadratic form. ( the covariance is actually a bilinear symmetric function ). Just plug your sum and use the independance of the variables, you will find that the sum contribute only to a factor $k$ while the leading $1/k$ contribute to $1/k^2$.

  • $\begingroup$ I think it would improve your answer to 1) define quadratic form, 2) explicitly point out that the independence assumption makes the variance additive. $\endgroup$ Jul 6 '15 at 12:29

This is a general property of averaging processes and not specific to the normal distribution. If you have independent and identically distributed random variables $X_1, X_2, \ldots , X_n$ with $\text{Var}(X_1) = \sigma^2 < \infty$ then,

$$ \begin{align} \text{Var} \left ( \frac{\sum_{i=1}^{n} X_i}{n} \right ) &= \frac{1}{n^2} \sum_{i=1}^{n} \text{Var}(X_i) \\ &= \frac{n \sigma^2}{n^2} \\ &= \frac{\sigma^2}{n} \end{align} $$

This makes sense if we think of each observation as a measurement of some average value $\mu$ plus an error term: $X_i = \mu + \epsilon_i$, where $\text{E}(\epsilon_i) = 0$. If we average up enough of these, we expect the error terms to roughly cancel and for the average to become increasingly concentrated around $\mu$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.