Say I have a slow MCMC program that samples from my model's posterior distribution. Assume further that there is no problem submitting this program as a job to a computer cluster multiple times. Each submission generates an independent realization of the Markov Chain of length $M$. Say for each job $j$, I can also calculate $\bar{\theta}_j = \sum_{i=1}^M \theta_j^i/M$.

$\bar{\theta} = \sum_{j=1}^J \bar{\theta}_j/J$ is still a consistent estimator, but what are all reasons this is an undesirable strategy? How do we get around plotting ACFs? We can just throw all the data from all jobs into one histogram, right? What types of people would object to this? Are there any mathematical difficulties I'm not thinking of? Where is this most commonly done, or who are doing this most often?

  • $\begingroup$ This is often what is done in practice, am I missing something? There is nothing wrong with this approach. In fact several convergence diagnostics require multiple independent chains (e.g. Gelman Rubin) $\endgroup$ – bdeonovic Feb 26 '17 at 23:56
  • $\begingroup$ @bdeonovic you are not. I don't see it as much, so I wanted to ask. Which convergence diagnostics? $\endgroup$ – Taylor Feb 27 '17 at 0:01
  • 1
    $\begingroup$ You haven't specified the initial condition that you're using for the different runs. Do you randomly pick the starting points? You also haven't talked about allowing the chain to burn-in. This strategy has the fault that you need to use a burn-in period on each of the separate runs. $\endgroup$ – Brian Borchers Feb 27 '17 at 0:03
  • $\begingroup$ @BrianBorchers, I wanted to abstract that detail away. Atm I'm just starting them all from the same arbitrarily chosen spot. And yes, that's right; no way around that. $\endgroup$ – Taylor Feb 27 '17 at 0:07
  • 1
    $\begingroup$ @Taylor Gelman-Rubin requires multiple independent chains. You should start the chains at different initial points (that is the benefit of running multiple chains: you don't have to worry about your results being dependent on your starting points). $\endgroup$ – bdeonovic Feb 27 '17 at 0:10

When running several Markov chains in parallel, the starting values of those chains start to matter. In short, if they are generated from $\mu$, the distribution of the chain after $n$ steps is $K^n\mu$. Rather than the target $\pi$. It is thus necessary to cull the burnin or warmup stage of the chain, which is much harder than with a single chain when one can invoke the Birkhof Theorem.

To decide where to cull can be done by coupling the chains, i.e., by checking approximately when a value from one chain can be accepted as following value by another chain, which should happen with probability were both chains in their stationary regime.

One way to check for convergence is to take the final part of each Markov chain, subsample drastically, and check for all components or some components of interest of the simulated parameter whether or not the distributions are the same, using for instance a Kolmogorov-Smirnov test.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.