I am designing a Hybrid Monte Carlo sampling algorithm for PyMC, and I am trying to make it as fuss free and general as possible, so I am looking for good advice on designing an HMC algorithm. I have read Radford's survey chapter and Beskos et. al.'s recent paper on optimal (step size) tuning of HMC and I gathered the following tips:

• Momentum variables should be distributed with covariance C^-1, where C is generally something like the covariance matrix of the distribution (for simple distributions), but could conceivably be different (for funny shaped distributions). By default I am using the hessian at the mode.
• Trajectories should be calculated with the leapfrog method (other integrators don't seem to be worth it)
• Optimal acceptance rate is .651 for really large problems and otherwise higher.
• Step size should be scaled like L * d^(1/4), where L is a free variable and d is the number of dimensions.
• Step size should be smaller when there are light tails or otherwise regions with odd stability characteristics. Step size randomization can help with this.

Are there other ideas that I should adopt or at least consider? Other papers I should read? For example, are there adaptive step size algorithms that are worth it? Is there good advice on trajectory length? Are there in fact better integrators?

Someone please make this a community-wiki.

I am designing a Hybrid Monte Carlo sampling algorithm for PyMC, and I am trying to make it as fuss free and general as possible, so I am looking for good advice on designing an HMC algorithm. I have read Radford's survey chapter and Beskos et. al.'s recent paper on optimal (step size) tuning of HMC and I gathered the following tips:

• Momentum variables should be distributed with covariance $C^{-1}$, where $C$ is generally something like the covariance matrix of the distribution (for simple distributions), but could conceivably be different (for funny shaped distributions). By default I am using the hessian at the mode.
• Trajectories should be calculated with the leapfrog method (other integrators don't seem to be worth it)
• Optimal acceptance rate is .651 for really large problems and otherwise higher.
• Step size should be scaled like $L\times d^{(1/4)}$, where $L$ is a free variable and $d$ is the number of dimensions.
• Step size should be smaller when there are light tails or otherwise regions with odd stability characteristics. Step size randomization can help with this.

Are there other ideas that I should adopt or at least consider? Other papers I should read? For example, are there adaptive step size algorithms that are worth it? Is there good advice on trajectory length? Are there in fact better integrators?

Someone please make this a community-wiki.