I have 2 normally distributed random variable $H_0$ and $H_1$, which are combined to give the weighted distribution $H$ as follows:

$H_0 \sim N(\mu_0, \sigma_0)$

$H_1 \sim N(\mu_1, \sigma_1)$

$$f_H = p * f_1(x) + (1-p) * f_0(x),$$

where $H$ has pdf $f_H$ and $H_1$ and $H_0$ have pdfs $f_1$ and $f_0$ respectively. The mean value $\mu$ of the combined distribution $H$ is:

$\mu = p * \mu_1 + (1-p) * \mu_0$

Now, what is the standard deviation $\sigma$ of $H$?

Is there a simple formula for $\sigma$, in terms of $p, \mu_0, \sigma_0, \mu_1, \sigma_1$?


(I am assuming $H_1$ and $H_0$ are independent)

Let $f_H = pf_1(x) + (1-p)f_0(x)$, where $f_1$ and $f_0$ are pdfs of $H_1$ and $H_0$. Then the random variable $H$ is the mixture of two normal distributions. For the mean of $H$

$$E(H) = \int x\left(pf_1(x) + (1-p)f_0(x) \right) dx = p\mu_1 + (1-p)\mu_0. $$

Similarly for the second moment of H \begin{align*} E(H^2)& = \int x^2 \left(pf_1(x) + (1-p)f_0(x) \right) dx\\ & = pE(H_1^2) + (1-p)E(H_0)^2\\ & = p(\sigma_1^2 + \mu_1^2) + (1-p)(\sigma_0^2 + \mu_0^2) \end{align*}

Finally, \begin{align*} Var(H) & = E(H^2) - [E(H)]^2\\ & = p(\sigma_1^2 + \mu_1^2) + (1-p)(\sigma_0^2 + \mu_0^2) - \left[p\mu_1 + (1-p)\mu_0 \right]^2\\ & = \left[p\sigma_1^2 + (1-p)\sigma_0^2\right] + [p\mu_1^2 + (1-p)\mu_0^2]- \left[p\mu_1 + (1-p)\mu_0 \right]^2\\ & = p\sigma^2_1+(1−p)\sigma^2_0+p(1−p)(\mu_1−\mu_0)^2 \end{align*}

Taking the square root of $Var(H)$, you get the standard deviation.

  • $\begingroup$ But why does $\sigma$ not depend on $\mu_0$ and $\mu_1$, and the difference between them? Surely $\sigma$ will be greater if the two means are further apart, no? $\endgroup$ – Kelvin Apr 2 '16 at 17:10
  • $\begingroup$ @Kelvin you are adding two normally distributed (independent) random variables, which is different from a mixture of Normally distributed random variables. $\endgroup$ – Greenparker Apr 2 '16 at 17:18
  • 1
    $\begingroup$ I have fixed the solution now. $\endgroup$ – Greenparker Apr 2 '16 at 17:35
  • 1
    $\begingroup$ Yes, I believe you are correct. $\endgroup$ – Greenparker Apr 2 '16 at 18:01
  • 1
    $\begingroup$ It's not homework, but maybe the simplified formula would help anyone else that reads this. $\endgroup$ – Kelvin Apr 2 '16 at 18:14

Let $Z$ denote a Bernoulli random variable with parameter $p$. Then, the random variable $H$ can be thought of as having conditional density $N(\mu_i, \sigma_i^2)$ according as $Z$ equals $i$, $i=0,1$, and thus unconditional density $$f_H(x) = pf_{H_1}(x) + (1-p)f_{H_0}(x).$$ The unconditional mean is thus the weighted sum of conditional means, viz. $$\mu_H = p\mu_1 + (1-p)\mu_0$$ while the unconditional (or total) variance is given by the total variance formula:

mean of the conditional variance plus the variance of the conditional mean

The first quantity is clearly $p\sigma_1^2 + (1-p)\sigma_0^2$. On the other hand, the conditional mean takes on value $\mu_1$ and $\mu_0$with probabilities $p$ and $1-p$ respectively and thus has variance \begin{align}p\mu_1^2 + (1-p)\mu_0^2 - \left(p\mu_1 + (1-p)\mu_0\right)^2 &= (p-p^2)\mu_1^2 + (1-p-(1-p)^2)\mu_0^2 -2p(1-p)\mu_0\mu_1\\ &= p(1-p)(\mu_1-\mu_0)^2 \end{align} Adding these two quantities together, we have that $$\sigma_H^2 = p\sigma_1^2 + (1-p)\sigma_0^2 + p(1-p)(\mu_1-\mu_0)^2$$ as in Greenparker's answer.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.