# What if Markov chain does not converge in a reasonable amount of time?

I'm doing data analysis using Hamiltonian Monte Carlo for sampling from the posterior distribution of weights of a neural network. I'm using the Gelman-Rubin diagnostic estimated potential scale reduction (ESPR) for checking the convergence of my Markov chains. My neural network has around 317 model weights and I check the convergence of each of the 317 parameters separately.

If I have understood everything correctly the parameters should have converged if the ESPR value for each of them is < 1.1.

This indeed does happen in most of the parameters but some weights seem not to converge in a reasonable amount of time. Some take up to 100.000 or more samples until they converge, which takes too long time in my analysis.

My question is: "What is the appropriate way to proceed if the Markov chains do not converge in a reasonable amount of time? Do I just need to bite the bullet and wait for three months or so?"

• Out of curiosity, let's say after 100,000 samples all 317 model weights have converged. What are you planning to do after that? Because even after the 100,000 steps you are going to have to run the Markov chain for longer. – Greenparker Dec 21 '17 at 10:58
• After the 317 weights have converged I will use e.g. the last 10% of the samples to calculate the neural network weights and then do prediction with the net. I do not fully understand your question? Can you clarify what you mean? – jjepsuomi Dec 21 '17 at 11:08
• So you will wait for the chain to have converged, and then use the last 10% (which is the part that that hasn't converged) to do the estimation. My point is at 100,000 the chain has converged. Now after that point you are starting to get more representative samples, so you may need to then keep sampling to ensure you get a good estimate. See discussion on effective sample size here – Greenparker Dec 21 '17 at 11:28
• To answer your original question 1) I work in MCMC and am not a huge fan of using Gelman-Rubin mainly because it is somewhat handwavy for my taste. However, if you still want to use it, maybe try a multivariate Gelman-Rubin. 2) I would suggest first looking at a trace plot for the weights that are slowly converging. Maybe it isn't a problem of multimodality etc than Gelman-Rubin is not able to catch. 3) Maybe focus on the quality of the estimates obtained by analysing the variance in the estimates. HMC is known to converge fairly quickly usually. – Greenparker Dec 21 '17 at 11:33
• "reasonable time" is in the eyes of the beholder! $10^5$ does not sound outrageous for a problem of dimension 317. Note that Rao-Blackwellisation and parallelisation should improve estimation if not the computing time. I am also adverse to keep only a small portion of the simulations: using the whole batch will always decrease the variance. – Xi'an Dec 27 '17 at 9:14