MCMC: long burn in vs re-initialization of the chain? The developer of the well-known emcee package often gives this advice to help with chain convergence:


*

*Run a short (few hundred steps) chain

*Reinitialize all the walkers near the point with maximum log probability seen so far

*Return to step 1 a few times

*Then run your final chain starting where you ended up for your last run of step 1


My question is: is this any different than simply allowing the chain(s) to run for more steps and then discard a longer burn-in period? I.e., is this any different from a "normal" run with more burn-in steps?
 A: The difference with the standard burn-in step in MCMC is that the later is usually done blindly, as a fixed fraction of the overall number of iterations, e.g., 20%. Here the burn-in or warm-up step is more actively looking for a reasonable starting point, that is, one that is compatible with the target density. The performance of the approach however depends on the mixing behaviour of the MCMC chain. If it mixes quite slowly relative to the contemplated horizon of a few hundred iterations, the chains will have likely stayed within the attraction basin of the mode found in the previous round. It would be better to consider annealed or relaxed targets during the warm-up, where flatter targets consisting of powered densities (with powers less than one) or partial posteriors (using only a fraction of the data) would be run (and the value of the actual target density monitored nonetheless). The preliminary exploration would be thus more effective and prone to leave attraction basins.
A: If you have only one single chain (or if you want all your chains to be completely independent), then this procedure is not different from classical burn-in.
However, it can accelerate convergence if you allow your chains to interact. Start from a random position, and let all your chains run independently for $T$ steps. Then, set all the walkers of all the chains to the same position (the point with maximum log probability seen so far across all chains), and let them run again independently. Hence, the chain that happened to be the closest to the high likelihood area will "guide" the others towards this position.
One possible drawback of this procedure is that it might increase the risk of getting stuck in a local optimum.
