One approach to model comparison in a Bayesian framework uses a Bernoulli indicator variable to indicate which of two models is likely to be the "true model". When applying MCMC-based tools for fitting such a model, it is common to use pseudo-priors to improve mixing in the chains. See here for a very accessible treatment of why pseudo-priors are useful. Apparently, it is well known that the use of pseudo-priors does not affect the long-term behavior of the chains (and therefore inference from the model); it only affects the short-term mixing.

This question seeks intuition about how it can possibly be the case that the pseudo-priors do not affect inference from the model. In particular something must be wrong either with the understanding that I have expressed above or with the reasoning that I describe below. I want to know exactly what I am getting wrong.

Suppose that in reality Model 1 and Model 2 are roughly equally probable given the data. But suppose that Jane inadvertently chooses pseudo-priors for Model 1 that are far away from the region of high posterior density. Suppose that Jane's pseudo-priors for Model 2, by contrast, fall directly in the region of high posterior density. Now consider what happens to a chain as it wanders around. When it lands in Model 1, it might remain in Model 1 for a little while, but can jump to Model 2 fairly easily. When it lands it Model 2, the poorly chosen pseudo-priors on Model 1 make it difficult to jump back to Model 1. It sure seems to me that as a result, the chain will spend a lot more time in Model 2 than in Model 1, which would lead to invalid inference.

So which part of this all am I misunderstanding?

  • $\begingroup$ When contemplating two and only two models there is no advantage in jumping between models compared with running two parallel MCMC chains, one for each model, and deriving the Bayes factor or the posterior probability after getting the chains. (The formal answer to your question is that the pseudo-priors need correction in an importance sampling type of correction.) $\endgroup$
    – Xi'an
    Jun 19, 2017 at 12:47
  • $\begingroup$ Fair enough. My understanding is that one application of indicator variables is as a tool to compute the ratio of the Bayes factors. In any case, when Carlin & Chib (1995 J. Roy. Stat. Soc.) say that "the form [of the pseudo-prior] is irrelevant", they only mean that it does not affect the estimates of the parameters that are "inside" M1 and M2, and NOT that it is irrelevant to inference regarding which model is correct? $\endgroup$ Jun 19, 2017 at 14:22
  • $\begingroup$ And if Jane wanted to use the indicator approach, how can she choose adequate pseudo priors when when the joint posterior distribution of the parameters in each model is high-dimensional and doesn't correspond to any "pretty" probability distribution that is easy to write down as a pseudoprior? Or is this basically impossible to do? $\endgroup$ Jun 19, 2017 at 14:24
  • $\begingroup$ Carlin and Chib (1995) use pseudo-priors for computational efficiency, but their outcome, namely the Bayes factor, remains the same, it does not depend on the choice of the pseudo-priors. So no to your question, inference about which model is correct does not depend on the pseudo-priors. $\endgroup$
    – Xi'an
    Jun 19, 2017 at 18:39
  • $\begingroup$ Right, so what part of the third paragraph in my question is wrong? Sorry if I'm misunderstanding something really basic here, or if my question is unclear in a way that I don't appreciate. $\endgroup$ Jun 20, 2017 at 19:33


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.