1
$\begingroup$

I currently am trying to figure out what would be the best option to perform MCMC sampling for a model which may show some kind of pathological behavior for some parameter combinations. The concrete case is a recurrent neural network (RNN), that I am running until convergence and then try to fit some values at the convergence point using MCMC. However, for some parameter combinations, the model may not converge or may produce some activations which are invalid.

That is, the MCMC should only sample from regions, where the parameters are valid and completely ignore invalid parameters.

The best option would, of course, be to use the priors to only allow sampling from those regions. However, this would require me to know the actual geometry of the valid region. While I currently have some intuition about this geometry (e.g. I expect it to be connected), I cannot give an analytical description, so I cannot constrain sampling in the prior.

Nevertheless, I can easily detect invalid parameters, after I run the model. For example, I detect divergent behavior by setting a maximum number of iterations and setting a flag when this number is exceeded.

Is it somehow possible to use this information to constrain the parameter space that is explored during sampling? The idea I currently have in mind is to use the flags that indicate pathological behavior as observed variables in a Bernoulli distribution with very low probability. This should reduce the likelihood of the sampler by drawing samples from this region of the parameter space and cause it to jump back to the allowed region.

Would this approach work in general, or could this cause additional artifacts? Also has this approach or a similar one been tried before, such that I can cite any literature on this approach?

Update

I am currently using the Metroplis step method from pymc3. I also tried NUTS, but this turned out to be slower by a factor of about 10 compared to Metropolis. The reason is not yet clear, and I believe NUTS may speed up when I manage to constrain the problem correctly.

Improve this question
$\endgroup$
6
  • $\begingroup$ What type of MCMC method do you use? $\endgroup$
    – Jonas
    Commented Aug 24, 2018 at 11:17
  • $\begingroup$ @Jonas: I am using the Metropolis sampler from pymc3 (see update). $\endgroup$
    – LiKao
    Commented Aug 24, 2018 at 11:48
  • $\begingroup$ I just found this question (stats.stackexchange.com/questions/73885/…), which might actually answer this. However, I am not sure and also this does not provide any literature on this topic, so I am still leaving this question open. $\endgroup$
    – LiKao
    Commented Aug 24, 2018 at 12:48
  • $\begingroup$ Initial thought: try reparameterising/transforming parameters. I assume you've done the necessary tuning of the MH sampler, could be that you need better proposals that account for the geometry of the posterior; I'd be tempted to try stochastic gradient Langevin dynamics, provided your posterior is diff'ble -- the 'optimise then draw samples' strategy is similar to SGLD. Also, there's a big difference between genuinely having constrained parameters, e.g. a Bernoulli parameter in $[0,1]$ vs. having nasty regions of parameter space where the sampler breaks down and then trying to avoid them. $\endgroup$
    – Will
    Commented Aug 24, 2018 at 15:47
  • $\begingroup$ @Will: Ok, probably a misunderstanding. It is not so much about divergent samples (in the MCMC sense), but about the iterative model showing divergent behavior. The model itself runs in a loop until the value does not change and then stops, this does not necessarily happen for all parameters. As such it is not the sampler that breaks down at these places but rather the model itself becoming undefined. However, whereas for a Bernoulli parameter I can easily avoid these regions (just don't pick any values outside [0,1]) I cannot (yet) give an analytical description of the valid parameter space. $\endgroup$
    – LiKao
    Commented Aug 24, 2018 at 18:03

1 Answer 1

Reset to default
1
$\begingroup$

I worked on an uncertainty quantification problem with a similar restriction: not all spaces in the proposal space could provide valid results.

One thing you can do, if you can quantify your restrictions (say, $\theta_0$ must be less than $\theta_1$), would be to wrap that logic into a validity function (return true or false). then before evaluating likelihood for a given proposal, check the proposal against the validity function. If it fails, set log-likelihood under proposal to -inf.

You might also create a more nuanced transformation of the parameters, but tuning a proposal density for these transformed parameters may be a nightmare.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.