# Metropolis-Hastings algorithm for logarithmic probability density

Similar question to posted here: Metropolis-Hastings using log of the density however my question is around sampling a random number from a uniform distribution. I am following the steps outlined in Murphy: Machine Learning A Probabilistic Perspective on page 850 for Metropolis Hastings algorithm. The difference is I am working with log probability densities as they are large and negative, so exponentiating them effecitvely results in 0.

I have started by taking the log of step 5: $$log(\alpha) = log(P(x')) - log(P(x))$$ Assuming q is symmetric which it is in my case. Then to compute $$r$$, I am taking the lesser of $$log(1)$$ and $$log(\alpha)$$, so essentially if $$\alpha$$ is negative, I am assigning it to $$r$$, otherwise I am assigning $$r$$ as 0. Now for the issue: what do I do at step 6? The link above doesn't describe this. Obviously $$r$$ is either 0 or negative, and so it would always be less than $$u$$ and hence we always step where we are. Also, $$r$$ isn't bounded below - $$\alpha$$ could be very negative (though unlikely), so it is hard to redefine a uniform distribution range to sample from. Any ideas how to proceed from here?

• I think that if you took $log(\alpha)$ then you will have to take $log(u)$ to preserve inequality. Feb 13 at 1:53
• So draw a random number from 0 to 1, and then take the log of it? Feb 13 at 1:54
• Precisely! I didn't get to the point, but as I like samplers, I found this post umbertopicchini.wordpress.com/2017/12/18/… Feb 13 at 2:14
• @jassis Thanks for the help - I have managed to get it to work. Out of interest, I have written the code two ways. In one instance I exponentiate $log(\alpha)$ and then continue the method in the image in the OP, in a second method I do the whole method in log format, doing as you said above and taking the log of $u$. Out of curiosity, do you know if the two methods should be equivalent? Or if one would be better than the other? I suspect there may be a difference when comparing $u$ to an exponentiated $log(\alpha)$ vs comparing $log(\alpha)$ to $log(u)$ due to different scales? Feb 13 at 3:27
• Both methods give exactly the same answer, they are more than equivalent. Note also that the min step is unnecessary for the comparison. Feb 13 at 9:25

In log scale, here is what your pseudocode should read from step 5 onwards:

5. Compute acceptance probability

$$\log(\alpha) = \log \frac{p(x')}{p(x)} = \log p(x')- \log p(x).$$

Compute

$$\log(r) =\min(0, \log(\alpha))$$

6. Sample $$u \sim U(0,1)$$

7. Set new sample to

$$x^{s+1} = \begin{cases} x' \quad \text{if} \quad \log(u) < \log(r) \\ x^s \quad \text{if} \quad \log(u) \geq \log(r)\end{cases}$$

As you've correctly noted, because $$q$$ is symmetric, we have that $$q(x | x') = q(x' | x)$$ and so these cancel in your expression for $$\log (\alpha)$$.

For $$\log(r)$$, note that $$\log(r) = \log \min(1, \alpha) = \min(\log(1), \log(\alpha)).$$

Now that you have $$\log(r)$$ you just need to compare it to $$\log(u)$$ as we can log both sides of the inequalities in the decision rule.