# When is MCMC useful?

I am having trouble in understanding in which situation the MCMC approach is actually useful. I am going through a toy example from the Kruschke book "Doing Bayesian Data Analysis: A Tutorial with R and BUGS".

What I understood so far is that we need a target distribution which is proportional to $p(D|\theta)p(\theta)$ in order to have a sample of $P(\theta|D)$. However, it seems to me that once we have $p(D|\theta)p(\theta)$ we only need to normalize the distribution to get the posterior, and the normalization factor could be easily found numerically. So what are the cases when this is not possible?

• Suppose $\theta$ is not a scalar but instead is a vector $\boldsymbol\theta$ having 10000 dimensions. – Jan Galkowski Feb 10 '15 at 19:47
• My answer was a little terse. To get the constant, need to calculate $\int_{-\infty}^{\infty} p(D|\theta)p(\theta)$. Even in the scalar case, suppose $p(D|\theta)$ is really wonky so the integration is difficult to do, even numerically. Then you may want to use MCMC. – Jan Galkowski Feb 10 '15 at 20:02
• A word of caution from Alan Sokal: "Monte Carlo is an extremely bad method; it should be used only when all alternative methods are worst". Then he embarks on a long dicussion of MC methods. stat.unc.edu/faculty/cji/Sokal.pdf – Yair Daon Feb 10 '15 at 20:33
• @Yair: It sounds to me like Sokal is channeling Churchill. – cardinal Feb 10 '15 at 21:16
• When nothing else will work ... – kjetil b halvorsen Feb 10 '15 at 23:18

Monte Carlo integration is one form of numerical integration which can be much more efficient than, e.g., numerical integration by approximating the integrand with polynomials. This is especially true in high dimensions, where simple numerical integration techniques require large numbers of function evaluations. To compute the normalization constant $p(D)$, we could use importance sampling,

$$p(D) = \int \frac{q(\theta)}{q(\theta)} p(\theta)p(D \mid \theta) \, d\theta \approx \frac{1}{N} \sum_n w_n p(\theta_n)p(D \mid \theta_n),$$

where $w_n = 1/q(\theta_n)$ and the $\theta_n$ are sampled from $q$. Note that we only need to evaluate the joint distribution at the sampled points. For the right $q$, this estimator can be very efficient in the sense of requiring very few samples. In practice, choosing an appropriate $q$ can be difficult, but this is where MCMC can help! Annealed importance sampling (Neal, 1998) combines MCMC with importance sampling.

Another reason why MCMC is useful is this: We usually aren't even that interested in the posterior density of $\theta$, but rather in summary statistics and expectations, e.g.,

$$\int p(\theta \mid D) f(\theta) \, d\theta.$$

Knowing $p(D)$ does not generally mean we can solve this integral, but samples are a very convenient way to estimate it.

Finally, being able to evaluate $p(D \mid \theta)p(\theta)$ is a requirement for some MCMC methods, but not all of them (e.g., Murray et al., 2006).

• Sorry, but this is still not clear to me. My question is: if we just multiply $p(D|\theta)p(\theta)$ we obtain a unnormalize pdf. By running MCMC we obtain a sample for which we can estimate the unnormalized pdf. If we want, we could normalize both. So, ASSUMING I am NOT interested in any summary statistics, but only in the posteriors, why we use MCMC in the first place? As you said, some MCMC methods do not require the computation of $p(D|\theta)p(\theta)$,soI am not referring to those. As far as I know, most of them require the computation of that. What is the usefulness of these methods? – Vaaal Feb 10 '15 at 19:38
• When running MCMC you get a sample from the normalised pdf so avoid computing the normalisation constant. And this is for free. – Xi'an Feb 10 '15 at 20:01
• @Vaaal: Your assumption that "the normalization factor could be easily found numerically" only holds for simple univariate distributions. For high-dimensional $\theta$, normalizing $p(D \mid \theta) p(\theta)$ is in general extremely difficult. In this case, MCMC can still be used to estimate the normalization constant (e.g. via annealed importance sampling). – Lucas Feb 10 '15 at 22:13

When you are given a prior $p(\theta)$ and a likelihood $f(x|\theta)$ that are either not computable in closed form or such that the posterior distribution $$p(\theta|x)\propto p(\theta)f(x|\theta)$$is not of a standard type, simulating directly from this target towards a Monte Carlo approximation of the posterior distribution is not feasible. A typical example is made of hierarchical models with non-conjugate priors, such as those found in the BUGS book.

Indirect simulation methods such as accept-reject, ratio-of-uniform, or importance-sampling techniques customarily run into numerical and precision difficulties when the dimension of the parameter $\theta$ increases beyond a few units.

On the opposite, Markov chain Monte Carlo methods are more ameanable to large dimensions in that they can explore the posterior distribution on a local basis, i.e. in a neighbourhood of the current value, and on a smaller number of components, i.e., on subspaces. For instance, the Gibbs sampler validates the notion that simulating from a one-dimensional target at a time, namely the full conditional distributions associated with $p(\theta|x)$, is sufficient to achieve simulation from the true posterior in the long run.

Markov chain Monte Carlo methods also some degree of universality in that algorithms like the Metropolis-Hastings algorithm is formally available for any posterior distribution $p(\theta|x)$ that can be computed up to a constant.

In cases when $p(\theta)f(x|\theta)$ cannot be easily computed, alternatives exist, either by completing this distribution into a manageable distribution over a larger space, as in$$p(\theta)f(x|\theta)\propto \int g(z|\theta,x) p(\theta)f(x|\theta)\text{d}z$$ or through non-Markovian methods like ABC.

MCMC methods have given a much broader reach for Bayesian methods, as illustrated by the upsurge that followed the popularisation of the method by Alan Gelfand and Adrian Smith in 1990.

• Link to THE BUGS BOOK isn't working anymore. – SmallChess Jun 11 '15 at 4:00