I am confused by the special way required to use inverse method in the following problem,

Here is the problem:

Consider a mixture distribution of two normal distributions, where the desired PDF $f(x)$ is given by:

$f(x) = r\, f_a(x) + (1 − r)\, f_b(x)$,where $f_a$ and $f_b$ are normal PDFs with means $a$ and $b$, respectively (standard deviation is 1 for both). Using two uniform random variables $u_1$ and $u_2$, explain how we can use the inversion method to sample from $f(x)$. Note, the qnorm command in R may be helpful here.

My confusion is from "two uniform random variables $u_1$ and $u_2$". My thought is that we find out the cdf, $F(x)$ (which can be obtained via pnorm() in R), and then we can use some numerical method (such as Newton-Raphson) to generate $x\sim f(x)$, so here it only needs one uniform distribution and does not need qnorm().

What's wrong with my method? Does the problem suggest a better method?

  • 2
    $\begingroup$ The effort involved in your approach is far greater (and unless special care is taken, it's more likely to suffer numerical stability issues). $\endgroup$ – Glen_b Oct 17 '13 at 2:50
  • $\begingroup$ If I want to simulate a mixture normal, for example: $$ r\times\mathcal{N}(a, 1) + (1-r)\times\mathcal{N}(b,1)$$ on the interval [0,1] instead of $\mathbb{R}$. Can I inspire the algorithm of @ Xi'an by simulating the truncated normal of $X_a$ or $X_b$ on [0,1]? $\endgroup$ – Alexy Mar 23 '15 at 13:11

The "two uniforms" are not absolutely necessary when generating from a mixture, but they make the simulation easy to understand. The mixture of normal distributions, $$rf_a(x)+(1-r)f_b(x)$$ has a probability mass of $r$ associated with the first normal and $(1-r)$ with the second normal. This means that the distribution of $X\sim f$ can be decomposed as $$\mathbb{P}(X\in\mathcal{A})=r\mathbb{P}(X_a\in\mathcal{A})+(1-r)\mathbb{P}(X_b\in\mathcal{A})$$ for any measurable set $\mathcal{A}$, where $X_a$ and $X_b$ are normal random variables with means $a$ and $b$ respectively. This can be reinterpreted as $$X=\begin{cases} X_a &\text{with probability $r$}\\ X_b &\text{with probability $1-r$}\end{cases}$$ meaning that to generate from the mixture, one can follow the steps

  1. Pick between components $a$ and $b$ by generating a uniform $U\sim\mathcal{U}(0,1)$ and, if $U<r$ take $\mu=a$ and else take $\mu=b$;
  2. Generate $X$ as $X_a$ or $X_b$ depending on the first step result, by generating a uniform $V\sim\mathcal{U}(0,1)$ and take $X=\Phi^{-1}(V)+\mu$

This explains for the use of two uniforms.

  • 6
    $\begingroup$ +1. The solution can be implemented with a single uniform $U$: when $U\le r$, return $\Phi^{-1}(U/r)+a$, otherwise return $\Phi^{-1}((1-U)/(1-r))+b$. $\endgroup$ – whuber Dec 16 '14 at 18:13

Is this the problem from the STA511 class?:)

pnorm() won't give you the right result, because it's a CDF. What you are looking is an inverse of the CDF, so you have to use qnorm() to get it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.