What are some well known improvements over textbook MCMC algorithms that people use for bayesian inference?

When I'm coding a Monte Carlo simulation for some problem, and the model is simple enough, I use a very basic textbook Gibbs sampling. When it's not possible to use Gibbs sampling, I code the textbook Metropolis-Hastings I've learned years ago. The only thought I give to it is choosing the jumping distribution or its parameters.

I know there are hundreds and hundreds of specialized methods that improve over those textbook options, but I usually never think about using/learning them. It usually feels like it's too much effort to improve a little bit what is already working very well.

But recently I've been thinking if maybe there aren't new general methods that can improve over what I've been doing. It's been many decades since those methods were discovered. Maybe I'm really outdated!

Are there any well known alternatives to Metropolis-Hastings that are:

• reasonably easy to implement,
• as universally appliable as MH,
• and always improves over MH's results in some sense (computational performance, accuracy, etc...)?

I know about some very specialized improvements for very specialized models, but are there some general stuff everybody uses that I don't know?

• Do you mean Markov chain Monte Carlo? The textbook improvements to Monte Carlo simulations that I can think of involve antithetic and/or stratified sampling, as well as quasi-Monte Carlo. Your mentioning of only Gibbs and Metropolis-Hastings is indicative of Bayesian computing, though. Commented Dec 26, 2012 at 17:59
• @StasK, Yes, I'm mainly interested in bayesian models and statistical physics models (which is just bayesian inference on gibbs-like distributions p(x) = 1/Z exp(-E(x)/T) ). Sorry for failing to mention that. Commented Dec 26, 2012 at 18:01
• (+1) OK, a nice general-purpose adaptive algorithm "recently" published and already implemented in R, Python and Matlab is the twalk. It works well but it is always a good practice to double-check using another method. BTW, MH is implemented in the R package mcmc. Of course, there are many others but most of them are not implemented in this level of generality and/or they are difficult to implement. Another popular area nowadays is Sequential Monte Carlo. I hope this helps.
– user10525
Commented Dec 26, 2012 at 18:20
• You may already know this, but slice sampling is pretty easy to implement and avoids some pitfalls of a typical "random-walk" Metropolis algorithm. A problem with traditional Metropolis algorithms is random-walk type behaviour; rather than move purposefully towards good states, they stumble around, moving slowly to the good areas. This is the motivation behind methods that make use of information in the gradient, such as HMC, but slice sampling also falls into this camp and is more automatic.
– guy
Commented Dec 26, 2012 at 21:54
• I know almost nothing about ABC, but I would be glad if anyone could explain a bit about the differences between ABC and MCMC methods... Commented Dec 31, 2012 at 16:45