7
$\begingroup$

I am confused by the special way required to use inverse method in the following problem,

Here is the problem:

Consider a mixture distribution of two normal distributions, where the desired PDF $f(x)$ is given by:

$f(x) = r\, f_a(x) + (1 − r)\, f_b(x)$,where $f_a$ and $f_b$ are normal PDFs with means $a$ and $b$, respectively (standard deviation is 1 for both). Using two uniform random variables $u_1$ and $u_2$, explain how we can use the inversion method to sample from $f(x)$. Note, the qnorm command in R may be helpful here.

My confusion is from "two uniform random variables $u_1$ and $u_2$". My thought is that we find out the cdf, $F(x)$ (which can be obtained via pnorm() in R), and then we can use some numerical method (such as Newton-Raphson) to generate $x\sim f(x)$, so here it only needs one uniform distribution and does not need qnorm().

What's wrong with my method? Does the problem suggest a better method?

$\endgroup$
2
  • 2
    $\begingroup$ The effort involved in your approach is far greater (and unless special care is taken, it's more likely to suffer numerical stability issues). $\endgroup$
    – Glen_b
    Oct 17, 2013 at 2:50
  • $\begingroup$ If I want to simulate a mixture normal, for example: $$ r\times\mathcal{N}(a, 1) + (1-r)\times\mathcal{N}(b,1)$$ on the interval [0,1] instead of $\mathbb{R}$. Can I inspire the algorithm of @ Xi'an by simulating the truncated normal of $X_a$ or $X_b$ on [0,1]? $\endgroup$
    – Alexy
    Mar 23, 2015 at 13:11

2 Answers 2

7
$\begingroup$

The "two uniforms" are not absolutely necessary when generating from a mixture, but they make the simulation easy to understand. The mixture of normal distributions, $$rf_a(x)+(1-r)f_b(x)$$ has a probability mass of $r$ associated with the first normal and $(1-r)$ with the second normal. This means that the distribution of $X\sim f$ can be decomposed as $$\mathbb{P}(X\in\mathcal{A})=r\mathbb{P}(X_a\in\mathcal{A})+(1-r)\mathbb{P}(X_b\in\mathcal{A})$$ for any measurable set $\mathcal{A}$, where $X_a$ and $X_b$ are normal random variables with means $a$ and $b$ respectively. This can be reinterpreted as $$X=\begin{cases} X_a &\text{with probability $r$}\\ X_b &\text{with probability $1-r$}\end{cases}$$ meaning that to generate from the mixture, one can follow the steps

  1. Pick between components $a$ and $b$ by generating a uniform $U\sim\mathcal{U}(0,1)$ and, if $U<r$ take $\mu=a$ and else take $\mu=b$;
  2. Generate $X$ as $X_a$ or $X_b$ depending on the first step result, by generating a uniform $V\sim\mathcal{U}(0,1)$ and take $X=\Phi^{-1}(V)+\mu$

This explains for the use of two uniforms.

$\endgroup$
1
  • 6
    $\begingroup$ +1. The solution can be implemented with a single uniform $U$: when $U\le r$, return $\Phi^{-1}(U/r)+a$, otherwise return $\Phi^{-1}((1-U)/(1-r))+b$. $\endgroup$
    – whuber
    Dec 16, 2014 at 18:13
0
$\begingroup$

Is this the problem from the STA511 class?:)

pnorm() won't give you the right result, because it's a CDF. What you are looking is an inverse of the CDF, so you have to use qnorm() to get it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.