6
$\begingroup$

Is it OK to choose the proposal distribution as the prior in a Metropolis algorithm?

Perhaps it's a simple question and to me, it's totally fine but as I always see people choose different distributions, I'm curious to see what others think.

$\endgroup$

4 Answers 4

8
$\begingroup$

It was not completely clear whether you meant the same distribution, but centered on the previous value of the chain or independent proposals drawn from the prior. Either of the two are no problem per-se, but are potentially inefficient.

Ideally, you would have independent draws from the posterior as your proposals, but that is of course in practice infeasible. Compared to that a proposal that is much wider more dispersed (which would often be the case for a prior, whether it is centered on the previous value of the chain or not) typically leads to a very low acceptance rate.

$\endgroup$
6
$\begingroup$

Besides the inefficiency of using the prior pointed out in other answers, there is one specific setting where one cannot use the prior distribution as proposal. This is when the prior distribution $\pi$ is improper, since one cannot generate from an infinite mass measure. It is then possible to construct a Markov kernel with stationary measure $\pi$, but there is no clear advantage in proceeding this way.

$\endgroup$
5
$\begingroup$

I get what you're asking (and if I'm right: yes, it is ok), but I think the question is ill-formed.

In MCMC you use a Markov chain to wander over some state space, visiting points of the space with probability proportional to some target function. The chain wanders from point to point based on some proposal distribution.

The target function needs to be proportional to a probability density, but that's it. It's not necessarily connected to a choice of prior or likelihood in a Bayesian model. We fruitfully use MCMC to approximate integrals over posterior distributions in Bayesian statistics, but you could also use e.g. the Metropolis algorithm to approximate the expectation of the distribution with density proportional to the function

$$ f(x, y) = \exp \left\{-(1 - x)^{2} - 100(y - x^{2})^2 \right\}. $$

Notice I haven't specified a Bayesian model consisting of prior & likelihood; I've just given you some function that is proportional to a probability density.

So: yes, feel free to use the same proposal distribution as your prior, if you're performing MCMC on a posterior. The two are not meaningfully connected.

$\endgroup$
1
  • $\begingroup$ Prior and posterior are "meaningfully" connected by the likelihood function and Bayes' theorem. $\endgroup$
    – Xi'an
    Jun 25, 2023 at 7:29
1
$\begingroup$

You can choose any distribution you want as a proposal distribution, including, of course, the prior distribution. But there's an interesting thing:

Suppose the likelihood function $L(\theta)=P(x|\theta)$, and the PDF of the prior distribution of parameters is $\Phi(\theta)=P(\theta)$. Now we want to sample on the posterior distribution which is proportional to $T(\theta)=L(\theta)\Phi(\theta)$.

If you choose the prior distribution $\Phi(x)$ as our proposal distribution, when we now have sample $x$, the probability density to propose a new sample $x'$ is:

$$P(x'|x)=\Phi(x')$$

Under MH sampling approach, consider the ratio:

$$\frac{T(x')\Phi(x)}{T(x)\Phi(x')}=\frac{L(x')\Phi(x')\Phi(x)}{L(x)\Phi(x)\Phi(x')}=\frac{L(x')}{L(x)}$$

Then the probability to accept the new sample $x'$ is:

$$\min\{1,\frac{L(x')}{L(x)}\}$$

Or, under log probability, is:

$$\min\{0,\ln{L(x')}-\ln{L(x)}\}$$

So, if you use your prior distribution directly as your proposal distribution, you can use the likelihood function to calculate the acceptance ratio directly. This may be helpful in some specific cases.

$\endgroup$
2
  • $\begingroup$ Which "specific cases"? It is usually the likelihood function that proves a challenge, rather than the prior. $\endgroup$
    – Xi'an
    Jun 25, 2023 at 9:45
  • $\begingroup$ @Xi'an In each run, the calculations of PDF of prior and proposal are not needed. I guess this, at least can save a little time maybe? $\endgroup$
    – C.K.
    Jun 26, 2023 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.