# What are some good references for the Metropolis-Hastings algorithm?

I need to study the Metropolis-Hastings algorithm and its properties, like convergence criteria. What is a good book, paper, or website that explains it using simple terms, but without being trivial?

An excellent introductory paper is

Chib, Siddhartha, and Edward Greenberg. “Understanding the Metropolis-Hastings Algorithm.” The American Statistician, vol. 49, no. 4, 1995, pp. 327–335.

A masterful and concise discussion of the theory is

Tierney, Luke. “Markov Chains for Exploring Posterior Distributions.” The Annals of Statistics, vol. 22, no. 4, 1994, pp. 1701–1728.

• Thanks a lot. My principal target is to learn about convergence criteria, but i know only the base of Metropolis Hastings, thus all it's useful. May 31, 2014 at 22:35
• Start studying convergence with Tierney. An exhaustive treatment is found in Meyn and Tweedie probability.ca/MT
– Zen
May 31, 2014 at 23:18
• And what about simulated annealing with Metropolis Hastings? I have read this but what about the integration with Metropolis Hastings? May 31, 2014 at 23:26
• Robert and Casella's book discusses simulated annealing. amazon.com/Monte-Statistical-Methods-Springer-Statistics/dp/…
– Zen
Jun 1, 2014 at 2:39
• "Understanding..." link is broken. Feb 15, 2016 at 18:53

For a book that is not "heavy on the math", I'd recommend:

Go to Chapter 7.

R code is provided in the book, so you'll be able to play around with the examples and see, hands-on, the effects of changing the number of burn-ins and so on.

There is a very good paper by Christian Robert describing M-H algorithm in detail

Robert, C. P. (2015). The Metropolis-Hastings algorithm. arXiv preprint arXiv:1504.01896.

and great book about Monte Carlo methods in general from the same author

Robert, C., & Casella, G. (2013). Monte Carlo statistical methods. Springer Science & Business Media.

With regards to convergence criteria most work is on convergence is the Total Variation (TV) distance sense. Mostly because there is a lot of probability theory worked out for TV distance. There is a nice survey paper and also on the theoretical side there is the paper by Roberts and Rosenthal that gives several theorems on convergence criteria. On the more practical side there are several papers written by Jim Hobert that provide examples of applying one of the theorems in Roberts and Rosenthal to MCMC. In general the tricky part of applying that theorem seems to be coming up with a good Lyapunov drift function.