I see a lot of examples using MCMC to solve for posterior distribution when the likelihood is simply one of linear regression. What if the likelihood is an ugly, complex function. Which R package can solve a problem like that?

It is a Gibbs sampler problem, because there are a number of RVs involved, and must be sampled in turn within one sweep.


Do they really have to be sampled in turn? There are MCMC algorithms that deal with high dimensional problems really well and in fact move through the parameter space much more efficiently than Gibbs sampling. In particular I am thinking of Hamiltonian Monte Carlo (HMC).

rstan may be your answer, since you can specify any log-likelihood and it deals with high dimensional problems really well by using the No-U-turn HMC sampler.

  • $\begingroup$ That's a whole new world you just got me into. I have a physics background and really appreciate the beauty of the Hamiltonian concept. Do you have recommendations as to going with r or python interface? $\endgroup$ – bhomass Jun 17 '17 at 22:00
  • $\begingroup$ I have only ever used the R interface, because I was already familiar with R and it's available at my company. It is well designed, but slightly complicated to install (primarily how the Rtools incl. the C++ compiler need to be set up, once that's done it's great). I cannot comment on the python interface, but I suspect it should be your first choice, if you're already a python person. $\endgroup$ – Björn Jun 18 '17 at 7:14
  • $\begingroup$ I went thru a lot of the Stan tutorial. I think at the end, it still not rich enough for my use case. I not only have a hmm involved, it is a dynamic hmm, where the transition probability is dynamically updated according to the coefficient values being gibbs sampled. Are you aware of such capabilities in Stan? $\endgroup$ – bhomass Jun 19 '17 at 0:54
  • $\begingroup$ Do you mean you have a density for each parameter conditional on all others and no way to specify it differently? Or something else? E.g. ordinary differential equations is something can do. You could also ask in the community forum (see mc-stan.org) the specific case. $\endgroup$ – Björn Jun 19 '17 at 5:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.