I'm using the Metorpolis-Hastings algorithm in a setting where the acceptance function is essentially of the form $$\alpha(x,y)=1\wedge\frac{u(x,y)}{v(x,y)},$$ where $$u(x,y)=p+(1-p)\prod_{i=1}^mu_i(x,y)$$ and $$v(x,y)=p+(1-p)\prod_{j=1}^nv_j(x,y).$$

My problem is that, if $m$ or $n$ is large enough (in my application less than $15$ is sufficient), it may happen that the product in the definition of $u(x,y)$ or $v(x,y)$ is computed to $0$ (due to insufficient floating point precision) even when each $u_i(x,y)$ or $v_i(x,y)$ is strictly positive (each $u_i(x,y)$ or $v_i(x,y)$ is essentially the value of a normal distribution density).

What can I do to remedy this issue?

EDIT: The concrete shape of $u$ and $v$ is $u(x,y)=w_{d_1}(\varphi(x),\varphi(y))$ and $v(x,y)=w_{d_2}(\psi(x),\psi(y))$, where $\varphi$ and $\psi$ are transformations onto $[0,1)^{d_1}$ and $[0,1)^{d_2}$, respectively, and $$w_d(x',y'):=p+\frac{1-p}{\sqrt{2\pi\sigma^2}}\prod_{i=1}^d\sum_{k\in\mathbb Z}e^{-\frac{\left(k+y'_i-x'_i\right)^2}{2\sigma^2}}.$$

  • 5
    $\begingroup$ Use the log-exp-sum trick, there are many posts on this site dealing with the issue. $\endgroup$ – hejseb Mar 13 at 18:45
  • 1
    $\begingroup$ What is the issue? The underflow of a product does not change the precision with which $\alpha$ is computed and for this purpose you need very little precision anyway--maybe three decimal places would suffice. $\endgroup$ – whuber Mar 13 at 19:22
  • 1
    $\begingroup$ When they are "very small," then unless $p$ itself is comparably small, it makes no difference. The only values of $p$ that would make a real difference would have to be less than $10^{-300}.$ $\endgroup$ – whuber Mar 13 at 19:29
  • 1
    $\begingroup$ There's a mismatch between your example and the formulas in your question: in the question, each of $u$ and $v$ starts with the value $p,$ to which another term is added. You are concerned that the additional term is rounded down to zero. That makes no difference unless the amount of rounding is appreciable relative to the size of $p.$ However, no additive term comparable to $p$ appears in your comment: what gives? $\endgroup$ – whuber Mar 13 at 19:47
  • 3
    $\begingroup$ If $p>0$, you have no problem for the reason given by whuber. If $p=0$, compute the products instead as sums in log-space, then subtract one from the other and take the exp() to get your $\alpha$. $\endgroup$ – Creosote Mar 13 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.