I'm looking for difficult Bayesian inference problems to test out different Monte Carlo sampling methods. I've mostly been looking at Hamiltonian Monte Carlo based algorithms and in particular, I've been using the trick of using the hessian at the Maximum A Posteriori point for the mass matrix.

What are some practical inference problems with difficult posteriors to sample from?

  • $\begingroup$ I think genetics, dynamical systems among other areas present challenges in sampling from the posterior of the parameters. For this reason ABC is becoming popular. The posterior of the parameters of Stable distributions is another example of difficult posteriors to sample from as well. Is this the sort of examples you are looking for? $\endgroup$ – user10525 May 9 '12 at 18:56
  • $\begingroup$ I think so, though more specific problems/datasets would be helpful as well as problems that are relatively simple to set up (I could get them running in a couple of hours on my home computer). $\endgroup$ – John Salvatier May 9 '12 at 19:59
  • $\begingroup$ The posteriors are often complicated in hierarchical models because the normalization constants of the first stage priors are included in the posteriors. $\endgroup$ – Stéphane Laurent May 10 '12 at 4:45
  • $\begingroup$ @John Salvatier, you don't mention Girolami and Calderhead, 2011 in the question -- I assume you've looked at the example inference problems in that paper? $\endgroup$ – Cyan May 10 '12 at 4:50
  • 1
    $\begingroup$ @Procrastinator, John Salvatier is looking for problems where the unnormalized target density can be computed for a given state, but has a shape that makes MCMC convergence difficult. ABC is for problems so difficult that the unnormalized target density can't be computed for a given state, making standard MCMC impossible. $\endgroup$ – Cyan May 10 '12 at 4:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.