Suppose that we want to sample from a posterior distribution $p(\theta|y)$ but we do not know how to directly sample. Suppose instead that we have a working set of values $\{\theta^{(1)}, \ldots, \theta^{(S)}\}$ and that we have a new draw $\theta^{*}$ that we want to know if we want to set equal to $\theta^{(S+1)}$, the next value of the set.

Then, the algorithm states to look at:

$$ r = \frac{p(\theta^{*}|y)}{p(\theta^{S}|y)} $$

If $r>1$, then $\theta^{*}$ is in a sense more likely than $\theta^{(S)}$, so we set $\theta^{*} = \theta^{(S+1)}$.

Now, if $r<1$, then $\theta^{*}$ is in a sense less likely than $\theta^{(S)}$. Then, we set $\theta^{*} = \theta^{(S+1)}$ with probability $r$ and $\theta^{*} = \theta^{(S)}$ with probability $1-r$.

This is where I am confused, if $p(\theta^{*}|y) < p(\theta^{S}|y)$, shouldn't we just directly reject $\theta^{*}$ since it is less likely to occur? Why do we even have this scheme where we will accept it with probability $r$? Why not directly remove it? Thanks.

  • 1
    $\begingroup$ Because you want to sample from the distribution, not just find the (local) maximum. Also, your $r$ equation is only for a symmetric proposal. $\endgroup$ – jaradniemi Mar 24 '17 at 0:51

If you reject lower probability regions, then you're basically just greedily hill-climbing. The problem with this is that you'll probably end up in a local maximum, and be unable to escape. If you're hunting for the global maximum, this is bad, since you're stuck in a local. If you want to sample from the distribution, this is bad, since even if you end up in the actual global maximum, you'll just sample the same point, the global maximum, over and over again, rather than sampling from the distribution.

So, you need to have the possibility of moving to lower probability areas, and through them, just that the probability of doing so is lower than the probability of moving to higher probability areas, which is what we want.


Not the answer you're looking for? Browse other questions tagged or ask your own question.