Hamiltonian Monte Carlo (HMC) seems like a powerful technique for sampling from probability distributions. However it seems that for it to be applicable, the parameter space has to be 'unconstrained', which makes it unclear how you would apply HMC to distributions involving the Von Mises Fisher distribution, or any other type of distribution on the unit sphere. What's the best way to use HMC to sample from distributions involving the Von Mises Fisher distribution?
2 Answers
If you search for "constrained HMC", there are a few papers on this general topic, see e.g. http://proceedings.mlr.press/v22/brubaker12.html.
More specifically, there is also work on "Geodesic Monte Carlo", which applies to highly-structured constrained spaces (e.g. spheres, orthogonal matrices); see https://arxiv.org/abs/1301.6064 for this.
In case of compositional data (which are reducible to a distribution on sphere by taking a square root) that satisfy:
$$
\sum_{N=1}^Nx_n=1
$$
one can use multinomial logit transformation:
$$
x_n=\frac{e^{u_n}}{\sum_{n=1}^{N-1}e^{u_n}+1}\text{ for } n<N,\\
x_N=\frac{1}{\sum_{n=1}^{N-1}e^{u_n}+1},
$$
whereas the Jacobian is
$$
\frac{\partial(x_1,...,x_{n-1})}{\partial(u_1,...,u_{n-1})}=\prod_{n=1}^Nx_n.$$