Metropolis-Hastings algorithms used in practice

Question

I was reading Christian Robert's Blog today and quite liked the new Metropolis-Hastings algorithm he was discussing. It seemed simple and easy to implement.

Whenever I code up MCMC, I tend to stick with very basic MH algorithms, such as independent moves or random walks on the log scale.

What MH algorithms do people routinely use? In particular:

• Why do you use them?
• In some sense you must think that they are optimal - after all you use them routinely! So how do you judge optimality: ease-of-coding, convergence, ...

I'm particularly interested in what is used in practice, i.e. when you code up your own schemes.

Answer 1

Hybrid Monte Carlo is the standard algorithm used for neural networks. Gibbs sampling for Gaussian process classification (when not using a deterministic approximation instead).

Answer 2

MH sampling is used when it's difficult to sample from the target distribution (e.g., when the prior isn't conjugate to the likelihood). So you use a proposal distribution to generate samples and accept/reject them based on the acceptance probability. The Gibbs sampling algorithm is a particular instance of MH where the proposals are always accepted. Gibbs sampling is one of the most commonly used algorithm due to its simplicity but it may not always to possible to apply, in which case one resorts to MH based on accept/reject proposals.

Answer 3

I use a slice sampler - originally proposed by Neal(2003), which I tune through heuristic optimization.

Answer 4

In physics, statistical physics in particular, Metropolis-type algorithm(s) are used extensively. There are really countless variants of these, and the new ones are being actively developed. It's much too broad topic to give any sort of expanation here, so if you're interested you can start e.g. from these lecture notes or from the ALPS library webpage (http://alps.comp-phys.org/mediawiki).