Does the MCMC algorithm use the likelihood function to move to a new proposal value?

Question

I'm trying to understand how MCMC and related algorithms work for Bayesian inference. In this paper the authors use an example of a normal distribution in explaining MCMC. They state,

"If the new proposal has a higher posterior value than the most recent sample, then accept the new proposal."

I have two questions on this:

1. Is this 'decision' to move based on the likelihood function?
2. I often read about 'ugly' distributions for priors and likelihoods (e.g. here). If a likelihood function is being use to move around, how can it do so if the distribution is ugly? My understanding is that if we want to apply a likelihood function we need the pdf as an argument.

Hope you can clarify, thanks.

Answer 1

Most MCMC algorithms do require the posterior density to be available up to a normalising constant, which means the likelihood $\ell(\theta_t|x)$ to be available as well in the sense that it can be computed for the actual data $x$ and an arbitrary value of the parameter $\theta$. For instance, the Metropolis-Hastings algorithm makes a proposal$$\theta'\sim q(\theta|\theta_t)$$and takes $\theta_{t+1}=\theta'$ with probability$$1 \wedge \dfrac{\pi(\theta')\ell(\theta'|x)q(\theta_t|\theta')}{\pi(\theta_t)\ell(\theta_t|x)q(\theta'|\theta_t)}$$When the Metropolis-Hastings ratio is larger than one (1) the proposed value is accepted with probability $1$.

The decision to accept (or to move) is therefore based on the value of the target density at the proposed and current values of the parameter. This usually depends in particular on the values of the likelihood function at the proposed and current values of the parameter.

Concerning the "ugliness" of the likelihood function, by which I assume a complex or costly function, provided the values of the likelihood function at the proposed and current values of the parameter can be computed in a reasonable time, this does not matter.

Situations when the values of the likelihood function at the proposed and current values of the parameter need not be computed include

1. the Gibbs sampler, which does not require an acceptance step
2. specific choices of the proposal $q(\cdot|\cdot)$ so that the ratio $$\dfrac{\pi(\theta')\ell(\theta'|x)q(\theta_t|\theta')}{\pi(\theta_t)\ell(\theta_t|x)q(\theta'|\theta_t)}$$does not require the full computation of the values of the likelihood function at the proposed and current values of the parameter (as for instance in Murray et al., 2012, resolution of doubly intractable problems)
3. replacements of the likelihood function with an unbiased estimator as in latent variable models (as in pseudo-marginal algorithms).