10
$\begingroup$

I am reading an awesome introductory HMC paper by Prof. Michael Betancourt, but getting stuck in understanding how do we go about the choice of the distribution of the momentum.

Summary

The basic idea of HMC is to introduce a momentum variable $p$ in conjunction with the target variable $q$. They jointly form a phase space.

The total energy of a conservative system is a constant and the system should follow Hamilton's equations. Therefore, the trajectories in the phase space can be decompose into energy levels, each level corresponds to a given value of energy $E$ and can be described as a set of points that satisfies:

$H^{-1}(E) = \{(q, p) | H(q, p) = E\}$.

We would like to estimate the joint distribution $\pi(q, p)$, so that by integrating out $p$ we get the desired target distribution $\pi(q)$. Furthermore, $\pi(q, p)$ can be equivalently written as $\pi(\theta_E \hspace{1.5pt} | \hspace{1.5pt} E) \hspace{1.5pt} \pi(E)$, where $E$ corresponds to a particular value of the energy and $\theta_E$ is the position on that energy level.

\begin{equation} \pi(q, p)= \begin{cases} \pi(p \hspace{1.5pt} | \hspace{1.5pt} q) \hspace{1.5pt} \pi(q) \\ \pi(\theta_E \hspace{1.5pt} | \hspace{1.5pt} E) \hspace{1.5pt} \pi(E), \hspace{5pt} \text{microcanonical decomposition} \end{cases} \end{equation}

For a given value of $E$, $\pi(\theta_E \hspace{1.5pt} | \hspace{1.5pt} E) $ is relatively easier to know, as we can perform integration of the Hamilton's equations to get the data points on the trajectory. However, $\pi(E)$ is the tricky part that depends on how we specify the momentum, which consequently determines the total energy $E$.

enter image description here

Questions

It seems to me that what we are after is $\pi(E)$, but what we can practically estimate is $\pi(E \hspace{1pt} | \hspace{1pt} q)$, based on the assumption that $\pi(E \hspace{2pt} | \hspace{1pt} q)$ can be approximately similar to $\pi(E)$, as illustrated in Fig. 23 of the paper. However, what we are actually sampling seems to be $\pi(p \hspace{1pt} | \hspace{1pt} q)$.

Q1: Is that because once we know $\pi(p \hspace{1pt} | \hspace{1pt} q)$, we can easily calculate $E$ and therefore estimate $\pi(E \hspace{2pt} | \hspace{1pt} q)$?

To make the assumption that $\pi(E) \sim \pi(E | q)$ hold, we use a Gaussian distributed momentum. Two choices are mentioned in the paper:

\begin{equation} \pi(p|q)= \begin{cases} \mathcal{N}(p \hspace{1pt}| \hspace{1pt} 0, M) \hspace{5pt} \text{Euclidean-Gaussian kinetic energy} \\ \mathcal{N}(p \hspace{1pt}| \hspace{1pt} 0, \Sigma(q)) \hspace{5pt} \text{Reimannian-Gaussian kinetic energy}, \end{cases} \end{equation}

where $M$ is a $D \times D$ constant called Euclidean metrics, aka mass matrix.

In the case of first choice (Euclidean-Gaussian), the mass matrix $M$ is actually independent of $q$, so the probability we are sampling is actually $\pi(p)$. The choice of the Gaussian-distributed momentum $p$ with covariance $M$ implies that the target variable $q$ is Gaussian-distributed with covariance matrix $M^{-1}$, as $p$ and $q$ need to be transformed inversely to keep the volume in the phase space constant.

Q2: My question is how can we expect $q$ to follow a Gaussian distribution? In practice $\pi(q)$ could be any complicated distribution.

$\endgroup$
11
$\begingroup$

It's not so much that we are after $\pi(E)$, it's just that if $\pi(E)$ and $\pi(E|q)$ are dissimilar then our exploration will be limited by our inability to explore all of the relevant energies. Consequently, in practice empirical estimates of $\pi(E)$ and $\pi(E|q)$ are useful for identifying any potential limitations of our exploration which is the motivation for the comparative histogram and the E-BFMI diagnostic.

So, what do we know about the two distributions? As we increase the dimensionality of our target distribution then $\pi(E)$ sort-of-tends to look more and more Gaussian. If our integration times are long enough then our explorations of the level sets will equilibrate and if $\pi(p | q)$ is Gaussian then $\pi(E|q)$ will also tend to be more and more Gaussian.

Hence a Gaussian-Euclidean kinetic energy is a good starting point but it is by no means always optimal! I spend a good bit of time trying to fit models where Stan yells at me about bad E-BFMI diagnostics. A Gaussian-Riemannian kinetic energy can be a significant improvement in many cases as the position-dependent log determinant in $\pi(p | q)$ can make $\pi(E)$ significantly more Gaussian, but this there is still much more research to be done to fully understand the problem.

$\endgroup$
4
  • 2
    $\begingroup$ I was trying to write an answer, but if THE Michael Betancourt is on Cross Validated, then I'll gladly step back :-) just a note, " log determent" is most likely a typo: I bet you meant "log determinant". $\endgroup$
    – DeltaIV
    Jan 3 '18 at 18:16
  • 1
    $\begingroup$ Flattered to have @Michael Betancourt's answer :-) Just curious, when you said "As we increase the dimensionality of our target distribution then $\pi(E)$ sort-of-tends to look more and more Gaussian", is there a theoretical proof or is it an empirical observation? $\endgroup$
    – cwl
    Jan 4 '18 at 1:58
  • 4
    $\begingroup$ @cwl - it's a standard asymptotic argument. If the target distribution converges to an independently distributed distribution with increasing data or parameters then $pi(E)$ will converge to a $chi^{2}$ which will be well approximated by a Gaussian above a few dimensions. On the other hand we tend to reach for Hamiltonian Monte Carlo in practice when the target is complex so the asymptotics may not be particularly relevant. Hence the hedging. $\endgroup$ Jan 4 '18 at 16:49
  • $\begingroup$ Got it @Michael Betancourt, thanks a lot for the explanation! $\endgroup$
    – cwl
    Jan 5 '18 at 1:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.