3
$\begingroup$

I have two independent gaussian distributions to combine and I have a doubt.

Let's say we have $X \sim N(\mu_x,\sigma^2_x)$ and $Y \sim N(\mu_y,\sigma^2_y)$. I want to mix the two variables with a 0.5 weight each, so I obtain a new variable $Z=0.5*X+0.5*Y$ with mean $\mu_z=0.5*\mu_x+0.5*\mu_y$.

Is it right to say that $\sigma^2_z=0.5^2*\sigma^2_x+0.5^2*\sigma^2_y$?

Because I clearly obtain less variance than intuitively expected when combining distributions.

Improve this question
$\endgroup$
7
  • $\begingroup$ Linear combination is very different than a mixture. If you are interested in a linear combination, your formula is correct. $\endgroup$
    – JohnK
    Jan 20, 2016 at 14:41
  • $\begingroup$ Sorry, I was refering with mixture as a linear combination with sum of weights=1 (I think someone thought that to me in a course), not as a mixture of random variable. That being said, isn't that strange that the variance reduces? I've never thought of this problem before. $\endgroup$
    – Arpayon
    Jan 20, 2016 at 14:45
  • $\begingroup$ Your intuition is "biased", i.e., wrong exectation. Take the special case of Y having the same mean and variance as X, i.e., X and Y are i.i.d., As you take the average of more and more i.i.d. Gaussian random variables, the variance of that average will decrease. Surely you are familiar with that concept. That's what's happening here. $\endgroup$
    – Mark L. Stone
    Jan 20, 2016 at 14:54
  • $\begingroup$ Hi Mark. In my case the normal variables are truncated expected lifetimes for males and females. i.e. I have 22 years for 60 years old males and 24 years for 60 years old females. Now, if I want to find some sort of general population truncated expected lifetime it's intuitive that's 23 (if I use same weight for male and female populations). Is variance decreasing because the result cannot be outside the previous intervals? Am I guessing it right? $\endgroup$
    – Arpayon
    Jan 20, 2016 at 15:39
  • $\begingroup$ When you average (for instance, 2) independent random variables, an extreme value in one variable tends not to be matched with a similar direction extreme value in the other variable, hence making the average less extreme than the individual random variables, hence the decreased variance. $\endgroup$
    – Mark L. Stone
    Jan 20, 2016 at 15:56

2 Answers 2

Reset to default
2
$\begingroup$

This answer should be viewed as being a compilation of comments which resulted in the OP's confusion being resolved, and is not intended as being the most general or rigorous statement on the subject. Its goal is to aid interpretation.

Jeremias K's answer shows the formula for the variance of a weighted sum of random variables. In particular, taking the special case of all weights being $1/n$ for a weighted sum of n independent random variables, the variance of the weighted sum equals $1/n^2$ times the sum of the variances of the individual random variables. If the random variables are also identically distributed, then the variance of the weighted sum equals $1/n$ times the variance of an individual random variable.

As you take the average of more and more i.i.d. (Gaussian) random variables, the variance of that average will decrease, as seen from the preceding formulation, which is a result which should be quite familiar. Why does the variance of that average decrease vs. an individual random variable? Well, taking the case of 2 independent random variables, an extreme value in one variable tends not to be matched with a similar direction extreme value in the other variable, hence making the average less extreme than the individual random variables, hence the decreased variance. The magnitude of this effect increases as the number of random variables being averaged increases.

Improve this answer
$\endgroup$
0
1
$\begingroup$

If $X$ and $Y$ are independent, $Cov(X,Y) = 0$. Since also, $Var(aX+bY) = a^2Var(X) + b^2Var(Y) + 2abCov(X,Y)$, it is in fact right to calculate the variance as you did.

I am not sure I can give an answer to the part of your question about intuition. I cannot really follow your intuition as to why the variance from the combination should be larger. Can you elaborate?

Improve this answer
$\endgroup$
1
  • $\begingroup$ Yeah, my problem is about interpretation not formulas, sadly. $\endgroup$
    – Arpayon
    Jan 20, 2016 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.