# MCMC sampling from model with invalid regions (divergent / pathological behavior) in the parameter space

I currently am trying to figure out what would be the best option to perform MCMC sampling for a model which may show some kind of pathological behavior for some parameter combinations. The concrete case is a recurrent neural network (RNN), that I am running until convergence and then try to fit some values at the convergence point using MCMC. However, for some parameter combinations, the model may not converge or may produce some activations which are invalid.

That is, the MCMC should only sample from regions, where the parameters are valid and completely ignore invalid parameters.

The best option would, of course, be to use the priors to only allow sampling from those regions. However, this would require me to know the actual geometry of the valid region. While I currently have some intuition about this geometry (e.g. I expect it to be connected), I cannot give an analytical description, so I cannot constrain sampling in the prior.

Nevertheless, I can easily detect invalid parameters, after I run the model. For example, I detect divergent behavior by setting a maximum number of iterations and setting a flag when this number is exceeded.

Is it somehow possible to use this information to constrain the parameter space that is explored during sampling? The idea I currently have in mind is to use the flags that indicate pathological behavior as observed variables in a Bernoulli distribution with very low probability. This should reduce the likelihood of the sampler by drawing samples from this region of the parameter space and cause it to jump back to the allowed region.

Would this approach work in general, or could this cause additional artifacts? Also has this approach or a similar one been tried before, such that I can cite any literature on this approach?

Update

I am currently using the Metroplis step method from pymc3. I also tried NUTS, but this turned out to be slower by a factor of about 10 compared to Metropolis. The reason is not yet clear, and I believe NUTS may speed up when I manage to constrain the problem correctly.

• What type of MCMC method do you use? – Jonas Aug 24 '18 at 11:17
• @Jonas: I am using the Metropolis sampler from pymc3 (see update). – LiKao Aug 24 '18 at 11:48
• I just found this question (stats.stackexchange.com/questions/73885/…), which might actually answer this. However, I am not sure and also this does not provide any literature on this topic, so I am still leaving this question open. – LiKao Aug 24 '18 at 12:48
• Initial thought: try reparameterising/transforming parameters. I assume you've done the necessary tuning of the MH sampler, could be that you need better proposals that account for the geometry of the posterior; I'd be tempted to try stochastic gradient Langevin dynamics, provided your posterior is diff'ble -- the 'optimise then draw samples' strategy is similar to SGLD. Also, there's a big difference between genuinely having constrained parameters, e.g. a Bernoulli parameter in $[0,1]$ vs. having nasty regions of parameter space where the sampler breaks down and then trying to avoid them. – Will Aug 24 '18 at 15:47
• @Will: Ok, probably a misunderstanding. It is not so much about divergent samples (in the MCMC sense), but about the iterative model showing divergent behavior. The model itself runs in a loop until the value does not change and then stops, this does not necessarily happen for all parameters. As such it is not the sampler that breaks down at these places but rather the model itself becoming undefined. However, whereas for a Bernoulli parameter I can easily avoid these regions (just don't pick any values outside [0,1]) I cannot (yet) give an analytical description of the valid parameter space. – LiKao Aug 24 '18 at 18:03