Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Based on the little knowledge that I have on MCMC (Markov chain Monte Carlo) methods, I understand that sampling is a crucial part of the aforementioned technique. The most commonly used sampling methods are Hamiltonian and Metropolis.
Is there a way to utilise machine learning or even deep learning to construct a more efficient MCMC sampler?
Yes. Unlike what other answers state, 'typical' machine-learning methods such as nonparametrics and (deep) neural networks can help create better MCMC samplers.
The goal of MCMC is to draw samples from an (unnormalized) target distribution $f(x)$. The obtained samples are used to approximate $f$ and mostly allow to compute expectations of functions under $f$ (i.e., high-dimensional integrals) and, in particular, properties of $f$ (such as moments).
Sampling usually requires a large number of evaluations of $f$, and possibly of its gradient, for methods such as Hamiltonian Monte Carlo (HMC). If $f$ is costly to evaluate, or the gradient is unavailable, it is sometimes possible to build a less expensive surrogate function that can help guide the sampling and is evaluated in place of $f$ (in a way that still preserves the properties of MCMC).
For example, a seminal paper (Rasmussen 2003) proposes to use Gaussian Processes (a nonparametric function approximation) to build an approximation to $\log f$ and perform HMC on the surrogate function, with only the acceptance/rejection step of HMC based on $f$. This reduces the number of evaluation of the original $f$, and allows to perform MCMC on pdfs that would otherwise too expensive to evaluate.
The idea of using surrogates to speed up MCMC has been explored a lot in the past few years, essentially by trying different ways to build the surrogate function and combine it efficiently/adaptively with different MCMC methods (and in a way that preserves the 'correctness' of MCMC sampling). Related to your question, these two very recent papers use advanced machine learning techniques -- random networks (Zhang et al. 2015) or adaptively learnt exponential kernel functions (Strathmann et al. 2015) -- to build the surrogate function.
HMC is not the only form of MCMC that can benefit from surrogates. For example, Nishiara et al. (2014) build an approximation of the target density by fitting a multivariate Student's $t$ distribution to the multi-chain state of an ensemble sampler, and use this to perform a generalized form of elliptical slice sampling.
These are only examples. In general, a number of distinct ML techniques (mostly in the area of function approximation and density estimation) can be used to extract information that might improve the efficiency of MCMC samplers. Their actual usefulness -- e.g. measured in number of "effective independent samples per second" -- is conditional on $f$ being expensive or somewhat hard to compute; also, many of these methods may require tuning of their own or additional knowledge, restricting their applicability.
References:
Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo for expensive Bayesian integrals." Bayesian Statistics 7. 2003.
Zhang, Cheng, Babak Shahbaba, and Hongkai Zhao. "Hamiltonian Monte Carlo Acceleration using Surrogate Functions with Random Bases." arXiv preprint arXiv:1506.05555 (2015).
Strathmann, Heiko, et al. "Gradient-free Hamiltonian Monte Carlo with efficient kernel exponential families." Advances in Neural Information Processing Systems. 2015.
Nishihara, Robert, Iain Murray, and Ryan P. Adams. "Parallel MCMC with generalized elliptical slice sampling." Journal of Machine Learning Research 15.1 (2014): 2087-2112.
A method that could connect the two concepts is that of a multivariate Metropolis Hastings algorithm. In this case, we have a target distribution (the posterior distribution) and a proposal distribution (typically a multivariate normal or t-distribution).
A well known fact is that the further the proposal distribution is from the posterior distribution, the less efficient the sampler is. So one could imagine using some sort of machine learning method to build up a proposal distribution that matches better to the true posterior distribution than a simple multivariate normal/t distribution.
However, it's not clear this would be any improvement to efficiency. By suggesting deep learning, I assume that you may be interested in using some sort of neural network approach. In most cases, this would be significantly more computationally expensive than the entire vanilla MCMC method itself. Similarly, I don't know any reason that NN methods (or even most machine learning methods) do a good job of providing adequate density outside the observed space, crucial for MCMC. So even ignoring the computational costs associated with building the machine learning model, I cannot see a good reason why this would improve the sampling efficiency.
Machine Learning is concerned with prediction, classification, or clustering in a supervised or unsupervised setting. On the other hand, MCMC is simply concerned with evaluating a complex intergral (usually with no closed form) using probabilistic numerical methods. Metropolis sampling is definitely not the most commonly used approach. In fact, this is the only MCMC method not to have any probabilistic component. So ML would not inform anything with MCMC in this case.
Importance based sampling does require a probabilistic component. It is more efficient than Metropolis under some basic assumptions. ML methods can be used to estimate this probabilistic component if it dovetails with some assumptions. Examples might be multivariate clustering to estimate a complex high dimensional Gaussian density. I am not familiar with non-parametric approaches to this problem, but that could be an interesting area of development.
Nonetheless, ML stands out to me as a distinct step in the process of estimating a high dimensional complex probability model which is subsequently used in a numerical method. I don't see how ML really improves MCMC in this case.
There were some recent works in computational physics where the authors used the Restricted Boltzmann Machines to model probability distribution and then propose (hopefully) efficient Monte Carlo updates arXiv:1610.02746. The idea here turns out to be quite similar to the references given by @lacerbi in above.
In another attempt 1702.08586, the author explicitly constructed Boltzmann Machines which can perform (and even discover) the celebrated cluster Monte Carlo updates.
