3
$\begingroup$

I am learning MCMC for the purpose of doing Bayesian inference. In Andrieu, 2003, it is mentioned that:

... in order to obtain the best results out of this class of algorithms, it is important that we do not treat them as black boxes, but instead try to incorporate as much domain specific knowledge as possible into their design.

The article was written 15 years ago. I am aware that now we have packages like Stan and JAGS (in R) that relieve users from worry about how to set the parameters of various MCMC algorithms to achieve best mixing.

My question is: Is domain knowledge still relevant when doing Bayesian inference using MCMC? (Edit: Do we still need to use domain knowledge to tweak the various algorithms, like choosing the proposal distribution, or setting the tuning parameters?)

If your answer is yes, could you give one example of where domain knowledge had helped you achieving better results? The more specific, the better.

| cite | improve this question | |
$\endgroup$
  • 2
    $\begingroup$ Is this asking about using subject matter knowledge for defining the model (eg. setting prior distributions)? Or about tweaking MCMC algorithms based on knowledge about the model/posterior structure instead of eg just using Stan for everything? (Based on the title I thought this would be about the first, and it seems the answer is about that, too, but I think the question body is actually about the second) $\endgroup$ – Juho Kokkala Jun 27 '18 at 16:33
  • 1
    $\begingroup$ @JuhoKokkala It is about the latter. Thank you for bringing this up. $\endgroup$ – user5228 Jun 28 '18 at 2:21
1
$\begingroup$

MCMC algorithms have improving since Andrieu (2003), and we now have the NUTS sampler in Stan, which is automated so that it can be used in applied problems without the user needing to understand the underlying algorithm. It is probably true that some difficult cases will require custom algorithms to run efficiently, but there is a trade-off between computational efficiency and human thinking time. In regard to this issue, there is a great quote by the mathematician and philosopher Alfred Whitehead that is one of my favourite pearls of wisdom:

"It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking what we are doing. The precise opposite is the case. Civilisation advances by extending the number of important operations which we can perform without thinking about them. Operations of thought are like cavalry charges in a battle — they are strictly limited in number, they require fresh horses, and must only be made at decisive moments.

--- Whitehead (1911) An Introduction to Mathematics, pp. 45-46.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ I think the ability to use NUTS without any understanding of HMC has been overstated. Understanding the geometry of the posterior is important in getting a sense of when NUTS is going to work, and it is very often the case that a poorly designed model (which relates back to the actual problem one is interested in) is going to have a likelihood with a difficult geometry. NUTS is also dependent on the parameterization of the model, and naive parameterizations frequently result in a disaster. $\endgroup$ – guy Jun 28 '18 at 2:46
  • $\begingroup$ Fair enough; my point was that the ideal is to get to a point where we have automated algorithms that a statistical analyst can use in a wide class of Bayesian models without thinking about it. I think we are getting closer to that, but there will always be some nasty cases (particularly in cases where there are identification issues) where customisation may be necessary. $\endgroup$ – Ben Jun 28 '18 at 3:01
  • $\begingroup$ @guy: I completely agree with your point that the belief that automated solutions will work uniformly well across models is mistaken. This was the case with BUGS in the 90's and is now the case with Stan. These software are great to explore the posterior but without understanding the way MCMC works and how [easily] it can err, this is a good recipe for disaster. $\endgroup$ – Xi'an Jun 28 '18 at 6:27
  • $\begingroup$ @guy I would like to know about examples, or references if possible, of how a naive parameterization of NUTS could lead to disaster. I just want to get a grasp of what could go wrong and how we can avoid it. Thanks. $\endgroup$ – user5228 Jun 28 '18 at 22:28
  • $\begingroup$ @user5228 the STAN manual actually has quite a bit of discussion on this. A classic example is that it matters a great deal how one deals with the variance parameters in hierarchical models, which is solved by the "Matt trick." See here. $\endgroup$ – guy Jun 29 '18 at 0:10
0
$\begingroup$

I agreed that it is unclear on the focus of the initial inquiry given the usage of the term "mixture". However, I will also point out that some individuals may opt to use Bayesian models with non-informative priors not to incorporate prior information, but to get estimates that may be deemed more interpretable to their general audience. For example, the difference between frequentist versus the Bayesian model output interpretations. This would be an approach not based on prior domain knowledge though uses Bayesian modeling.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This does not seem to be an answer to the question (with the clarified scope) $\endgroup$ – Juho Kokkala Jun 28 '18 at 20:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.