# How does the random walk know the height of the posterior in Bayesian inference?

How does the sampling procedure work if the posterior that is being sampled from is unknown?

If the proposed jump has a higher probability than the current one the random walk jumps to the proposed site, but how is the probability assessed at any time?

## 1 Answer

You need to be able to evaluate the target density pointwise to run MH.

Suppose you want to obtain samples $x^{(s)} \sim \pi(x)$ and propose from a local random walk $q(x) = \mathcal{N}(x, \sigma)$. Then in order to accept/reject a step $x \to x^\star$ in a Metropolis-Hastings manner you need to evaluate:

$$1 \wedge \frac{\pi(x^\star)}{\pi(x)}$$

up to a multiplicative constant (since that'd be present in both the numerator and denumerator and thus drop out).

If you cannot evaluate $\pi(x)$ you cannot apply many MCMC methods. There are ways to do away with this requirement, but they carry no statistical guarantees and are rather advanced.

• Is there any chance you could expand upon your answer perhaps using an example? Thanks. – adkane Apr 15 '17 at 19:44
• @ManassaMauler added a bit more detail – Oxonon Apr 15 '17 at 20:26