I'm using Metropolis-Hastings to sample from an Inv-Gamma$(a,b)$ posterior distribution. My jumping distribution $J_t(\theta_*|\theta_{t-1})$ is N$(\theta_{t-1},0.5^2)$. After I sample a $\theta_*$ value from the jumping distribution, I compute the acceptance ratio $$r = \frac{p(\theta_*|y) J_t(\theta_{t-1}|\theta_*)}{p(\theta_{t-1}|y) J_t(\theta_*|\theta_{t-1})} = \frac{\theta_*^{-(a+1)} \exp(-b/\theta_*) \exp\left[-\frac{(\theta_{t-1}-\theta_{t-1})^2}{2(0.5)^2}\right]}{\theta_{t-1}^{-(a+1)} \exp(-b/\theta_{t-1}) \exp\left[-\frac{(\theta_*-\theta_{t-1})^2}{2(0.5)^2}\right]}$$ and proceed from there. However, I'm getting bad results. I think that $$J_t(\theta_{t-1}|\theta_*) = \exp\left[-\frac{(\theta_{t-1}-\theta_{t-1})^2}{2(0.5)^2}\right] = 1$$ is wrong but I don't know exactly why and I don't know how to correct it. Where did I go wrong?

  • $\begingroup$ As indicated in the answer below, the confusion is indeed in misinterpreting $J_t(\theta_{t-1}|\theta_*)$ which is a function of $\theta_{t-1}-\theta_*$ and further should not be indexed by $t$. $\endgroup$ – Xi'an Apr 5 '17 at 9:10

When you say $J_t(\theta_{t-1}|\theta_*)$ that means $\theta_{t-1}$ is normally distributed with mean $\theta_{*}$. The only way you can get this density equal to $1$ if you jump and stay in exactly the same place. This has probability $0$ because the normal distribution is continuous.

$$ J_t(\theta_{t-1}|\theta_*) \propto \exp\left[-\frac{(\theta_{t-1}-\theta_{*})^2}{2(0.5)^2}\right] \neq 1 \text{ with probability $1$} $$ $$ J_t(\theta_*|\theta_{t-1}) \propto \exp\left[-\frac{(\theta_{*}-\theta_{t-1})^2}{2(0.5)^2}\right] \neq 1 \text{ with probability $1$} $$ The normalizing constants will cancel out in the ratio, in this case.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.