I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far:

  1. By default JAGS uses a slice sampler. However, each update for a single variable of the slice sampler will take many calls to the likelihood for each Gibbs step. While the slice sampler is very good at mixing, the large number of calls makes it hard for me to get a sufficient number of samples.

  2. Since I wasn't satisfied with the standard univariate slice sampler, I implemented a multivariate Metropolis. This Multivariate Metropolis only requires a single computation of the likelihood and is therefore very fast, but at the same time it suffers from strong autocorrelation within the chains (bad $\hat{R}$ statistics and divergent chains).

So it seems the slice sampler and the multivariate Metropolis sampler are two extremes. Are there other algorithms, which are more in-between these two extremes and can provide computational efficiency (as few calls to the likelihood as possible) as well as good mixing of the chains?

  • 2
    $\begingroup$ It all depends on the nature of the complexity. For instance, for large datasets, delayed acceptance could bring gains. For almost intractable likelihoods, ABC-Gibbs could work. &tc. &tc. $\endgroup$
    – Xi'an
    Nov 5 '20 at 9:49
  • $\begingroup$ @Xi'an The high computation cost comes from computations that are used to determine distribution parameters. These parameters are based on a multidimensional ODE. So the sampled parameters are the parameters to the ODE and the solution of the ODE (at convergence) is then used to determine parameters for distributions for the data. The dataset isn't particularly large, but the ODE computation requires a lot of time. $\endgroup$
    – LiKao
    Nov 5 '20 at 11:15
  • $\begingroup$ The goal would then be to find a cheaper but reliable approximation to the true target, run an MCMC sampler on the approximation and propose simulated values as in an independent Metropolis sampler. $\endgroup$
    – Xi'an
    Nov 5 '20 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.