Inverse transformation sampling for mixture distribution of two normal distributions

Question

I am confused by the special way required to use inverse method in the following problem,

Here is the problem:

Consider a mixture distribution of two normal distributions, where the desired PDF $f(x)$ is given by:

$f(x) = r\, f_a(x) + (1 − r)\, f_b(x)$,where $f_a$ and $f_b$ are normal PDFs with means $a$ and $b$, respectively (standard deviation is 1 for both). Using two uniform random variables $u_1$ and $u_2$, explain how we can use the inversion method to sample from $f(x)$. Note, the qnorm command in R may be helpful here.

My confusion is from "two uniform random variables $u_1$ and $u_2$". My thought is that we find out the cdf, $F(x)$ (which can be obtained via pnorm() in R), and then we can use some numerical method (such as Newton-Raphson) to generate $x\sim f(x)$, so here it only needs one uniform distribution and does not need qnorm().

What's wrong with my method? Does the problem suggest a better method?

Answer 1

The "two uniforms" are not absolutely necessary when generating from a mixture, but they make the simulation easy to understand. The mixture of normal distributions, $$rf_a(x)+(1-r)f_b(x)$$ has a probability mass of $r$ associated with the first normal and $(1-r)$ with the second normal. This means that the distribution of $X\sim f$ can be decomposed as $$\mathbb{P}(X\in\mathcal{A})=r\mathbb{P}(X_a\in\mathcal{A})+(1-r)\mathbb{P}(X_b\in\mathcal{A})$$ for any measurable set $\mathcal{A}$, where $X_a$ and $X_b$ are normal random variables with means $a$ and $b$ respectively. This can be reinterpreted as $$X=\begin{cases} X_a &\text{with probability $r$}\\ X_b &\text{with probability $1-r$}\end{cases}$$ meaning that to generate from the mixture, one can follow the steps

1. Pick between components $a$ and $b$ by generating a uniform $U\sim\mathcal{U}(0,1)$ and, if $U<r$ take $\mu=a$ and else take $\mu=b$;
2. Generate $X$ as $X_a$ or $X_b$ depending on the first step result, by generating a uniform $V\sim\mathcal{U}(0,1)$ and take $X=\Phi^{-1}(V)+\mu$

This explains for the use of two uniforms.

Answer 2

Is this the problem from the STA511 class?:)

pnorm() won't give you the right result, because it's a CDF. What you are looking is an inverse of the CDF, so you have to use qnorm() to get it.