Assuming that I have two normal distributions $p_1(x)$ and $p_2(x)$ and I can draw samples efficiently from them.

Now, I can easily draw samples from $p(x) = 0.5 p_1(x) + 0.5 p_2(x)$ by with probability = 50% taking a sample from $p_1$ and probability = 50% taking a sample from $p_2$.

Is there any similar way that I can efficiently draw samples from a difference of two distributions? For example $p(x) = 1.5 p_1(x) - 0.5 p_2(x)$, assuming that $p(x) > 0$?

  • $\begingroup$ Note that your proposed method to sample from the positive mixture is incorrect, because in any random sample it is more likely than not that one of the mixture components is represented more than the other. Just consider the problem you run into for a sample size that is an odd number! Are you sure you are not confusing mixtures with linear combinations of random variables? The sum (or mean) of two normal variates does not have the average of the PDFs for its PDF. $\endgroup$ – whuber Nov 21 '14 at 17:31
  • $\begingroup$ I think I made a mistake when saying "with 50% taking..". I meant for each sample, I flip a coin and with probability = 50% I get that sample from $p_1(x)$ and with probability = 50% I take from $p_2(x)$. $\endgroup$ – mushin Nov 22 '14 at 9:06
  • $\begingroup$ Ok, that's a mixture and now you have correctly described how to draw a sample from it (which differs from the description in the question). But then what does "$1.5p_1(x)-0.5p_2(x)$" mean? You can't flip a coin having probabilities of $1.5$ and $-0.5$! $\endgroup$ – whuber Nov 22 '14 at 15:29

You can't do this in general, since there will typically be areas where the density is negative, making it not a valid pdf.

enter image description here

In cases where $p(x) = 1.5 p_1(x) - 0.5 p_2(x)$ is sure to be non-negative, it's at least a density, and you can generate samples from it, for example by accept-reject (e.g. using $1.5 p_1(x)$ as majorizing function).

| cite | improve this answer | |
  • $\begingroup$ Thanks! We do have the constraint that $p(x)$ is positive. I have thought about rejection sampling; however this is a bit inefficient. Thus, I was looking for some direct sampling strategy similar to the case of mixture distribution. $\endgroup$ – mushin Nov 22 '14 at 9:10
  • $\begingroup$ But it's not a mixture. The naive accept-reject I suggested will accept 2/3 of points, which is pretty decent. More efficient approaches (mostly more efficient forms of accept-reject) could be constructed for particular cases, but at best you can only go from 2/3 to 1 anyway. $\endgroup$ – Glen_b Nov 22 '14 at 9:26
  • $\begingroup$ I found a related problem here link. I think I will stick with rejection sampling. Thanks a lot! $\endgroup$ – mushin Nov 28 '14 at 17:29
  • $\begingroup$ If I understood that post correctly, the algorithm there is the same naive accept-reject with c=1.5 that I suggested above. $\endgroup$ – Glen_b Nov 29 '14 at 2:09
  • $\begingroup$ Yes, that is the same with your suggestion. Thanks $\endgroup$ – mushin Nov 29 '14 at 10:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.