Debugging MCMC programs is notoriously difficult. The difficulty arises because of several issues some of which are:
(a) Cyclic nature of the algorithm
We iteratively draw parameters conditional on all other parameters. Thus, if a implementation is not working properly it is difficult to isolate the bug as the issue can be anywhere in the iterative sampler.
(b) The correct answer is not necessarily known.
We have no way to tell if we have achieved convergence. To some extent this can be mitigated by testing the code on simulated data.
In light of the above issues, I was wondering if there is a standard technique that can be used to debug MCMC programs.
I wanted to share the approach I use to debug my own programs. I, of course, do all of the things that PeterR mentioned. Apart from those, I perform the following tests using simulated data:
Start all parameters from true values and see if the sampler diverges too far from the true values.
I have flags for each parameter in my iterative sampler that determines whether I am drawing that parameter in the iterative sampler. For example, if a flag 'gen_param1' is set to true then I draw 'param1' from its full conditional in the iterative sampler. If this is set to false then 'param1' is set to its true value.
Once I finish writing up the sampler, I test the program using the following recipe:
- Set the generate flag for one parameter to true and everything else to false and assess convergence with respect to true value.
- Set the generate flag for another parameter in conjunction with the first one and again assess convergence.
The above steps have been incredibly helpful to me.