# What if Markov chain does not converge in a reasonable amount of time?

I'm doing data analysis using Hamiltonian Monte Carlo for sampling from the posterior distribution of weights of a neural network. I'm using the Gelman-Rubin diagnostic estimated potential scale reduction (ESPR) for checking the convergence of my Markov chains. My neural network has around 317 model weights and I check the convergence of each of the 317 parameters separately.

If I have understood everything correctly the parameters should have converged if the ESPR value for each of them is < 1.1.

This indeed does happen in most of the parameters but some weights seem not to converge in a reasonable amount of time. Some take up to 100.000 or more samples until they converge, which takes too long time in my analysis.

My question is: "What is the appropriate way to proceed if the Markov chains do not converge in a reasonable amount of time? Do I just need to bite the bullet and wait for three months or so?"

• Out of curiosity, let's say after 100,000 samples all 317 model weights have converged. What are you planning to do after that? Because even after the 100,000 steps you are going to have to run the Markov chain for longer. Dec 21 '17 at 10:58
• After the 317 weights have converged I will use e.g. the last 10% of the samples to calculate the neural network weights and then do prediction with the net. I do not fully understand your question? Can you clarify what you mean? Dec 21 '17 at 11:08
• So you will wait for the chain to have converged, and then use the last 10% (which is the part that that hasn't converged) to do the estimation. My point is at 100,000 the chain has converged. Now after that point you are starting to get more representative samples, so you may need to then keep sampling to ensure you get a good estimate. See discussion on effective sample size here Dec 21 '17 at 11:28
• To answer your original question 1) I work in MCMC and am not a huge fan of using Gelman-Rubin mainly because it is somewhat handwavy for my taste. However, if you still want to use it, maybe try a multivariate Gelman-Rubin. 2) I would suggest first looking at a trace plot for the weights that are slowly converging. Maybe it isn't a problem of multimodality etc than Gelman-Rubin is not able to catch. 3) Maybe focus on the quality of the estimates obtained by analysing the variance in the estimates. HMC is known to converge fairly quickly usually. Dec 21 '17 at 11:33
• "reasonable time" is in the eyes of the beholder! $10^5$ does not sound outrageous for a problem of dimension 317. Note that Rao-Blackwellisation and parallelisation should improve estimation if not the computing time. I am also adverse to keep only a small portion of the simulations: using the whole batch will always decrease the variance. Dec 27 '17 at 9:14

1. I am not a huge fan of using Gelman-Rubin mainly because it is somewhat handwavy for my taste. However, if you still want to use it, maybe try a multivariate Gelman-Rubin since it is possible the joint posterior of the weights have a complicated dependence structure that the univariate diagnostic is not able to capture. See answer here.
2. I would suggest first looking at a trace plot for the weights that are slowly converging. Maybe it is a problem of multimodality etc than Gelman-Rubin is not able to catch.
3. HMC is known to convergence fairly quickly usually in many situations. Maybe focus on the quality of the estimates obtained by analysing the variance in the estimates. You can find a discussion of the methods here.
4. To actually improve converge of the chain, you can try different starting values for the slow converging chains. You can also may be tweak the HMC wherever possible. It is also possible that HMC just doesn't work here, and a variant of the Metropolis-Hastings algorithm might work better. I won't be able to say anything without knowing more about the problem.

This was touched on by @Greenparker but the easiest way to speed a slowly converging model is to pick the right starting values. This is counterintuitive, as if you knew the right parameter values in the first place why would you fit the model. But, if you can fit a similar frequentist model quickly, use those parameter estimates as starting values.

Again, this assumes that your chains are eventually converging. If they do not converge at all then you may have a redundant parameter problem such that there are multiple combinations of parameters that are equally plausible, resulting in chains that will never converge.