I'm using Metropolis-Hastings to sample from an Inv-Gamma$(a,b)$ posterior distribution. My jumping distribution $J_t(\theta_*|\theta_{t-1})$ is N$(\theta_{t-1},0.5^2)$. After I sample a $\theta_*$ value from the jumping distribution, I compute the acceptance ratio $$r = \frac{p(\theta_*|y) J_t(\theta_{t-1}|\theta_*)}{p(\theta_{t-1}|y) J_t(\theta_*|\theta_{t-1})} = \frac{\theta_*^{-(a+1)} \exp(-b/\theta_*) \exp\left[-\frac{(\theta_{t-1}-\theta_{t-1})^2}{2(0.5)^2}\right]}{\theta_{t-1}^{-(a+1)} \exp(-b/\theta_{t-1}) \exp\left[-\frac{(\theta_*-\theta_{t-1})^2}{2(0.5)^2}\right]}$$ and proceed from there. However, I'm getting bad results. I think that $$J_t(\theta_{t-1}|\theta_*) = \exp\left[-\frac{(\theta_{t-1}-\theta_{t-1})^2}{2(0.5)^2}\right] = 1$$ is wrong but I don't know exactly why and I don't know how to correct it. Where did I go wrong?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ As indicated in the answer below, the confusion is indeed in misinterpreting $J_t(\theta_{t-1}|\theta_*)$ which is a function of $\theta_{t-1}-\theta_*$ and further should not be indexed by $t$. $\endgroup$– Xi'anCommented Apr 5, 2017 at 9:10
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
When you say $J_t(\theta_{t-1}|\theta_*)$ that means $\theta_{t-1}$ is normally distributed with mean $\theta_{*}$. The only way you can get this density equal to $1$ if you jump and stay in exactly the same place. This has probability $0$ because the normal distribution is continuous.
$$ J_t(\theta_{t-1}|\theta_*) \propto \exp\left[-\frac{(\theta_{t-1}-\theta_{*})^2}{2(0.5)^2}\right] \neq 1 \text{ with probability $1$} $$ $$ J_t(\theta_*|\theta_{t-1}) \propto \exp\left[-\frac{(\theta_{*}-\theta_{t-1})^2}{2(0.5)^2}\right] \neq 1 \text{ with probability $1$} $$ The normalizing constants will cancel out in the ratio, in this case.
