4
$\begingroup$

Is there an efficient approach tailored to sampling from a PDF $k \cdot f(x) \cdot g(x)$ (where $k$ is a normalizing constant) that would perform better than naive Metropolis, slice sampling, etc.? Can we exploit the fact that both $f(x)$ and $g(x)$ can already easily be sampled from?

An example of this might be sampling $X \sim k \cdot f(x|a) \cdot f(e^x|b)$ where $f(x|\theta)$ is the normal distribution, and $a$ and $b$ are known. Note that the terms aren't independent, so Gibbs sampling wouldn't necessarily help (this is actually meant to be part of a Gibbs sampler).

$\endgroup$
0

1 Answer 1

3
$\begingroup$

Yes, there are other methods based on rejection sampling

Marrelec, G. and Benali, H. (2004). Automated rejection sampling from product of distributions. Statistics and Computing Volume 19, Issue 2, pp 301-315.

http://link.springer.com/article/10.1007%2FBF02892062

The authors claim that this method is efficient.

From there, we show that an automated selection of both the comparison function and the upper bound is possible. Moreover, this choice is performed in order to optimize the sampling efficiency among a range of potential solutions.

In my opinion, although this is a nice and ingenious method, if the $f$ and $g$ are not too complicated, then the use of MCMC algorithms might be easier to implement.

$\endgroup$
2
  • $\begingroup$ Thanks, that was very helpful. I was hoping to avoid rejection algorithms, but this one seems like it might have a decent acceptance ratio if $f(x)$ and $g(x)$ are close. My implementation of a toy example (similar to that in the paper) was actually just four lines of Python code! $\endgroup$
    – roger_
    Apr 14, 2013 at 23:01
  • 1
    $\begingroup$ Is that code open source?? 🙃 $\endgroup$ Dec 23, 2019 at 22:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.