I am recently running a Bayesian model based on DRAM (Delayed Rejection Adaptive Metropolis) sampling on R with FME package. As the analysis is consuming considerable time, I am planning to move it to a multicore computer. However, as what I understood, we are not able to parallelize MCMC code to speed up the computing process, or it will break its serial nature. The best we can do is to run multiple chains on multiple processors and summarize them afterwards.

I was therefore wondering:

  • Does it mean that we can speed up the computing process by reducing iterations to a low number for each chain and increasing the number of chains we run in total?
  • I looked up the High-Performance and Parallel Computing with R page and found that there are only packages for distributed computing of multiple MCMC chains using BUGS and JAGS. I wonder does it mean that it will be rather difficult to parallelly process the analysis with other Bayesian packages like FME?

As I haven't found much example code on this topic, I would also appreciate if anyone could direct me to the related discussions. Thank you very much in advance.

  • $\begingroup$ You should be able to run parallel chains using the method you are using now provided you have access to a multi core machine. Look into the parallel package. A better place for posing that question is Stack Overflow. $\endgroup$
    – HStamper
    Commented Apr 15, 2017 at 12:30
  • $\begingroup$ Thank you very much, @EricMittman. I thought if I run multiple chains with parallel package like multicore or snowfall, it will be difficult to summarize the chains afterwards. Maybe I should check again. $\endgroup$
    – CYH
    Commented Apr 18, 2017 at 13:28
  • 1
    $\begingroup$ Just an update. I found there seems to be a new package called BayesianTools that has parallel processing functions inbuilt. It supports more MCMC sampling methods and is easy to program as well. May worth trying out. $\endgroup$
    – CYH
    Commented Apr 18, 2017 at 13:33

1 Answer 1


Your understanding is mostly right, with the caveat that your MCMC sampler is able to converge to sampling from the distribution of interest in a reasonable time.

Lets say for your MCMC sampler it takes 100,000 iterations before the sampler has converged to sampling from the proper distribution. It would be better in this case to have 1 chain run for 150,000 iterations than having two chains run for 75,000 iterations.

Also, many convergence diagnostics require multiple chains to be run anyway.

Finally, if the distribution you are sampling from is highly multi-modal you may want several chains with different initial configurations so that you are able to traverse the sample space rather then getting stuck in a few modes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.