In Hamiltonian Monte Carlo the proposal is accepted with probability: $$ \alpha\left(\mathbf{x}_n(0),\mathbf{x}_n(L\Delta t)\right) = \min\left(1, \frac{\exp\left[-H\left(\mathbf{x}_n(L\Delta t),\mathbf{p}_n(L\Delta t)\right)\right]} {\exp\left[-H\left(\mathbf{x}_n(0),\mathbf{p}_n(0)\right)\right]}\right), $$ where $\mathbf{x}_n(L\Delta t),\mathbf{p}_n(L\Delta t)$ are obtained via Hamiltonian evolution from $\mathbf{x}_n(0),\mathbf{p}_n(0)$, i.e., by solving numerically the Hamilton equations-of-motion: $$ \frac{d}{dt}\mathbf{x}_n(t) = \nabla_{\mathbf{p}_n}H\left(\mathbf{x}_n( t),\mathbf{p}_n(t)\right)\\ \frac{d}{dt}\mathbf{p}_n(t) = -\nabla_{\mathbf{x}_n}H\left(\mathbf{x}_n( t),\mathbf{p}_n(t)\right). $$ If solving these equations exactly, the Hamiltonian function $H\left(\mathbf{x}_n( t),\mathbf{p}_n(t)\right)$ does not change in time (known as energy conservation in mechanics), which means that the acceptance probability is always $1$... but for the errors of numerical integration (besides the cited Wikipedia article, the same prescription is given in MCMC using Hamiltonian dynamics by Radford Neal and A Conceptual Introduction to Hamiltonian Monte Carlo by Michael Betancourt).

Not a bug, but a feature
Question Hamiltonian Monte Carlo: how to make sense of the Metropolis-Hasting proposal? raises the same point. @MichaelBetancourt in their answer and subsequent comments explain that, if we could solve the Hamiltonian equations-of-motion exactly, the acceptance probability of $1$ would be a good choice. In other words, the acceptance rejection step serves only to correct the integration error.

Metropolis-Hastings with a Hamiltonian step
On the other hand, one could think of a genuinely Metropolis-Hastings algorithm with a step consisting of randomly changing momentum and performing Hamiltonian evolution. The acceptance probability would be then something like $$ \alpha\left(\mathbf{x}_{n-1}(0),\mathbf{p}_{n-1}(0);\mathbf{x}_n(L\Delta t), \mathbf{p}_n(L\Delta t)\right) = \min\left(1, \frac{\exp\left[-H\left(\mathbf{x}_n(L\Delta t),\mathbf{p}_n(L\Delta t)\right)\right]} {\exp\left[-H\left(\mathbf{x}_{n-1}(0),\mathbf{p}_{n-1}(0)\right)\right]}\right), $$ (there may be obvious variations, depending on how the proposal for the initial momentum is constructed). The Hamiltonian evolution probably still allows to escape local minima, just as in the Hamiltonian Monte Carlo.

I am interested in the relative advantages/disadvantages of the two approaches (ability to escape local minima, convergence, speed), particularly in terms of their applicability to mixture models.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.