If I'm understanding rejection sampling correctly, it's a way for us to sample a distribution that is difficult to directly sample. In order to apply rejection sampling of a distribution $p(z)$, we set $p(z) = {1 \over Z_p}\tilde{p}(z)$. From there we create a proposal distribution $q(z)$ and find a constant $k$ such that $kq(z) \ge \tilde{p}(z)$.

What I'm confused about is $\tilde{p}(z)$. In the book (PR and ML by Bishop), it says that we can easily able to evaluate $p(z)$ for any given value of z up to some normalizing constant $Z$. What are some cases of this? I can't think of a reason why we need to use $\tilde{p}(z)$ when we know $p(z)$.

  • $\begingroup$ Sampling from the univariate truncated normal distribution: link.springer.com/article/10.1007%2FBF00143942?LI=true $\endgroup$ – user3903581 Jul 16 '17 at 8:54
  • $\begingroup$ There are many cases (as in Bayesian statistics) when the density to simulate is known up to a normalising constant. $\endgroup$ – Xi'an Mar 18 '18 at 17:21

Using $\tilde{p}(z)$ sometimes makes the math easier to work out because you can ignore the normalizing constant $Z$. One example is drawing from a beta(2,2) via rejection sampling. You can bound the density by 2, or you can drop the normalizing constant $\frac{\Gamma(4)}{\Gamma(2)\Gamma(2)}$ and work directly with the kernel, $p(1-p)$.

Or, in the notation of the OP, $\tilde{p}(z) = z(1-z)$ and $p(z) =z(1-z) \frac{\Gamma(4)}{\Gamma(2)\Gamma(2)}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.