Is it OK to choose the MH proposal as the prior in a posterior simulation problem?

Question

Is it OK to choose the proposal distribution as the prior in a Metropolis algorithm?

Perhaps it's a simple question and to me, it's totally fine but as I always see people choose different distributions, I'm curious to see what others think.

Answer 1

I get what you're asking (and if I'm right: yes, it is ok), but I think the question is ill-formed.

In MCMC you use a Markov chain to wander over some state space, visiting points of the space with probability proportional to some target function. The chain wanders from point to point based on some proposal distribution.

The target function needs to be proportional to a probability density, but that's it. It's not necessarily connected to a choice of prior or likelihood in a Bayesian model. We fruitfully use MCMC to approximate integrals over posterior distributions in Bayesian statistics, but you could also use e.g. the Metropolis algorithm to approximate the expectation of the distribution with density proportional to the function

$$ f(x, y) = \exp \left\{-(1 - x)^{2} - 100(y - x^{2})^2 \right\}. $$

Notice I haven't specified a Bayesian model consisting of prior & likelihood; I've just given you some function that is proportional to a probability density.

So: yes, feel free to use the same proposal distribution as your prior, if you're performing MCMC on a posterior. The two are not meaningfully connected.

Answer 2

Besides the inefficiency of using the prior pointed out in other answers, there is one specific setting where one cannot use the prior distribution as proposal. This is when the prior distribution $\pi$ is improper, since one cannot generate from an infinite mass measure. It is then possible to construct a Markov kernel with stationary measure $\pi$, but there is no clear advantage in proceeding this way.

Answer 3

It was not completely clear whether you meant the same distribution, but centered on the previous value of the chain or independent proposals drawn from the prior. Either of the two are no problem per-se, but are potentially inefficient.

Ideally, you would have independent draws from the posterior as your proposals, but that is of course in practice infeasible. Compared to that a proposal that is much wider more dispersed (which would often be the case for a prior, whether it is centered on the previous value of the chain or not) typically leads to a very low acceptance rate.

Answer 4

You can choose any distribution you want as a proposal distribution, including, of course, the prior distribution. But there's an interesting thing:

Suppose the likelihood function $L(\theta)=P(x|\theta)$, and the PDF of the prior distribution of parameters is $\Phi(\theta)=P(\theta)$. Now we want to sample on the posterior distribution which is proportional to $T(\theta)=L(\theta)\Phi(\theta)$.

If you choose the prior distribution $\Phi(x)$ as our proposal distribution, when we now have sample $x$, the probability density to propose a new sample $x'$ is:

$$P(x'|x)=\Phi(x')$$

Under MH sampling approach, consider the ratio:

$$\frac{T(x')\Phi(x)}{T(x)\Phi(x')}=\frac{L(x')\Phi(x')\Phi(x)}{L(x)\Phi(x)\Phi(x')}=\frac{L(x')}{L(x)}$$

Then the probability to accept the new sample $x'$ is:

$$\min\{1,\frac{L(x')}{L(x)}\}$$

Or, under log probability, is:

$$\min\{0,\ln{L(x')}-\ln{L(x)}\}$$

So, if you use your prior distribution directly as your proposal distribution, you can use the likelihood function to calculate the acceptance ratio directly. This may be helpful in some specific cases.