7
$\begingroup$

I have 2 normally distributed random variable $H_0$ and $H_1$, which are combined to give the weighted distribution $H$ as follows:

$H_0 \sim N(\mu_0, \sigma_0)$

$H_1 \sim N(\mu_1, \sigma_1)$

$$f_H = p * f_1(x) + (1-p) * f_0(x),$$

where $H$ has pdf $f_H$ and $H_1$ and $H_0$ have pdfs $f_1$ and $f_0$ respectively. The mean value $\mu$ of the combined distribution $H$ is:

$\mu = p * \mu_1 + (1-p) * \mu_0$

Now, what is the standard deviation $\sigma$ of $H$?

Is there a simple formula for $\sigma$, in terms of $p, \mu_0, \sigma_0, \mu_1, \sigma_1$?

$\endgroup$
1
  • $\begingroup$ See the relevant part of the mixture distribution page at wikipedia; also can be found here on CV, e.g. here $\endgroup$
    – Glen_b
    Commented Aug 11, 2016 at 2:43

2 Answers 2

11
$\begingroup$

(I am assuming $H_1$ and $H_0$ are independent)

Let $f_H = pf_1 + (1-p)f_0$, where $f_1$ and $f_0$ are pdfs of $H_1$ and $H_0$. Then the random variable $H$ is the mixture of two normal distributions. For the mean of $H$

$$E(H) = \int x\left(pf_1(x) + (1-p)f_0(x) \right) dx = p\mu_1 + (1-p)\mu_0. $$

Similarly for the second moment of H \begin{align*} E(H^2)& = \int x^2 \left(pf_1(x) + (1-p)f_0(x) \right) dx\\ & = pE(H_1^2) + (1-p)E(H_0^2)\\ & = p(\sigma_1^2 + \mu_1^2) + (1-p)(\sigma_0^2 + \mu_0^2) \end{align*}

Finally, \begin{align*} Var(H) & = E(H^2) - [E(H)]^2\\ & = p(\sigma_1^2 + \mu_1^2) + (1-p)(\sigma_0^2 + \mu_0^2) - \left[p\mu_1 + (1-p)\mu_0 \right]^2\\ & = \left[p\sigma_1^2 + (1-p)\sigma_0^2\right] + [p\mu_1^2 + (1-p)\mu_0^2]- \left[p\mu_1 + (1-p)\mu_0 \right]^2\\ & = p\sigma^2_1+(1−p)\sigma^2_0+p(1−p)(\mu_1−\mu_0)^2 \end{align*}

Taking the square root of $Var(H)$, you get the standard deviation.

$\endgroup$
11
  • $\begingroup$ But why does $\sigma$ not depend on $\mu_0$ and $\mu_1$, and the difference between them? Surely $\sigma$ will be greater if the two means are further apart, no? $\endgroup$
    – Kelvin
    Commented Apr 2, 2016 at 17:10
  • $\begingroup$ @Kelvin you are adding two normally distributed (independent) random variables, which is different from a mixture of Normally distributed random variables. $\endgroup$ Commented Apr 2, 2016 at 17:18
  • $\begingroup$ For example, if $\mu_0$ and $\mu_1$ were very different, but $\sigma_0$ and $\sigma_1$ were both zero, the combined standard deviation $\sigma$ would not be zero, would it? Something is not right here... $\endgroup$
    – Kelvin
    Commented Apr 2, 2016 at 17:20
  • 1
    $\begingroup$ I have fixed the solution now. $\endgroup$ Commented Apr 2, 2016 at 17:35
  • 1
    $\begingroup$ Yes, I believe you are correct. $\endgroup$ Commented Apr 2, 2016 at 18:01
4
$\begingroup$

Let $Z$ denote a Bernoulli random variable with parameter $p$. Then, the random variable $H$ can be thought of as having conditional density $N(\mu_i, \sigma_i^2)$ according as $Z$ equals $i$, $i=0,1$, and thus unconditional density $$f_H(x) = pf_{H_1}(x) + (1-p)f_{H_0}(x).$$ The unconditional mean is thus the weighted sum of conditional means, viz. $$\mu_H = p\mu_1 + (1-p)\mu_0$$ while the unconditional (or total) variance is given by the total variance formula:

mean of the conditional variance plus the variance of the conditional mean

The first quantity is clearly $p\sigma_1^2 + (1-p)\sigma_0^2$. On the other hand, the conditional mean takes on value $\mu_1$ and $\mu_0$with probabilities $p$ and $1-p$ respectively and thus has variance \begin{align}p\mu_1^2 + (1-p)\mu_0^2 - \left(p\mu_1 + (1-p)\mu_0\right)^2 &= (p-p^2)\mu_1^2 + (1-p-(1-p)^2)\mu_0^2 -2p(1-p)\mu_0\mu_1\\ &= p(1-p)(\mu_1-\mu_0)^2 \end{align} Adding these two quantities together, we have that $$\sigma_H^2 = p\sigma_1^2 + (1-p)\sigma_0^2 + p(1-p)(\mu_1-\mu_0)^2$$ as in Greenparker's answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.