Standard deviation for weighted sum of normal distributions [duplicate]

Question

I have 2 normally distributed random variable $H_0$ and $H_1$, which are combined to give the weighted distribution $H$ as follows:

$H_0 \sim N(\mu_0, \sigma_0)$

$H_1 \sim N(\mu_1, \sigma_1)$

$$f_H = p * f_1(x) + (1-p) * f_0(x),$$

where $H$ has pdf $f_H$ and $H_1$ and $H_0$ have pdfs $f_1$ and $f_0$ respectively. The mean value $\mu$ of the combined distribution $H$ is:

$\mu = p * \mu_1 + (1-p) * \mu_0$

Now, what is the standard deviation $\sigma$ of $H$?

Is there a simple formula for $\sigma$, in terms of $p, \mu_0, \sigma_0, \mu_1, \sigma_1$?

Answer 1

(I am assuming $H_1$ and $H_0$ are independent)

Let $f_H = pf_1 + (1-p)f_0$, where $f_1$ and $f_0$ are pdfs of $H_1$ and $H_0$. Then the random variable $H$ is the mixture of two normal distributions. For the mean of $H$

$$E(H) = \int x\left(pf_1(x) + (1-p)f_0(x) \right) dx = p\mu_1 + (1-p)\mu_0. $$

Similarly for the second moment of H \begin{align*} E(H^2)& = \int x^2 \left(pf_1(x) + (1-p)f_0(x) \right) dx\\ & = pE(H_1^2) + (1-p)E(H_0^2)\\ & = p(\sigma_1^2 + \mu_1^2) + (1-p)(\sigma_0^2 + \mu_0^2) \end{align*}

Finally, \begin{align*} Var(H) & = E(H^2) - [E(H)]^2\\ & = p(\sigma_1^2 + \mu_1^2) + (1-p)(\sigma_0^2 + \mu_0^2) - \left[p\mu_1 + (1-p)\mu_0 \right]^2\\ & = \left[p\sigma_1^2 + (1-p)\sigma_0^2\right] + [p\mu_1^2 + (1-p)\mu_0^2]- \left[p\mu_1 + (1-p)\mu_0 \right]^2\\ & = p\sigma^2_1+(1−p)\sigma^2_0+p(1−p)(\mu_1−\mu_0)^2 \end{align*}

Taking the square root of $Var(H)$, you get the standard deviation.

Answer 2

Let $Z$ denote a Bernoulli random variable with parameter $p$. Then, the random variable $H$ can be thought of as having conditional density $N(\mu_i, \sigma_i^2)$ according as $Z$ equals $i$, $i=0,1$, and thus unconditional density $$f_H(x) = pf_{H_1}(x) + (1-p)f_{H_0}(x).$$ The unconditional mean is thus the weighted sum of conditional means, viz. $$\mu_H = p\mu_1 + (1-p)\mu_0$$ while the unconditional (or total) variance is given by the total variance formula:

mean of the conditional variance plus the variance of the conditional mean

The first quantity is clearly $p\sigma_1^2 + (1-p)\sigma_0^2$. On the other hand, the conditional mean takes on value $\mu_1$ and $\mu_0$with probabilities $p$ and $1-p$ respectively and thus has variance \begin{align}p\mu_1^2 + (1-p)\mu_0^2 - \left(p\mu_1 + (1-p)\mu_0\right)^2 &= (p-p^2)\mu_1^2 + (1-p-(1-p)^2)\mu_0^2 -2p(1-p)\mu_0\mu_1\\ &= p(1-p)(\mu_1-\mu_0)^2 \end{align} Adding these two quantities together, we have that $$\sigma_H^2 = p\sigma_1^2 + (1-p)\sigma_0^2 + p(1-p)(\mu_1-\mu_0)^2$$ as in Greenparker's answer.

Answer 3

I don't have an account here, so I can't comment. But in the accepted answer, you need to square $pf_1(x)+(1-p)f_0(x)$ in your second integral as well. Thus, the variance is not a linear combination of the variance, it is the squared sum: $p^2 \sigma_1^2+(1-p)^2 \sigma_0^2+\dots$. For intuition, consider the simple case where $\mu_1=0$ and $\sigma_1=0$. You are just multiplying $H_0$ with a scalar $1-p$, which should give you variance of $(1-p)^2\sigma_0^2$, not $(1-p)\sigma_0^2$. See: http://theanalysisofdata.com/probability/2_3.html, https://en.wikipedia.org/wiki/Normal_distribution