One approach to model comparison in a Bayesian framework uses a Bernoulli indicator variable to determine which of two models is likely to be the "true model". When applying MCMC-based tools for fitting such a model, it is common to use pseudo-priors to improve mixing in the chains. See here for a very accessible treatment of why pseudo-priors are useful.
In their seminal paper on the topic, Carlin & Chib (p. 475) state that "the form of [the pseudo-prior] is irrelevant," which I take to mean that it should not affect posterior inference based on the model (though it might affect MCMC mixing during model fitting). However, my inuition is that the form of the pseudo-prior DOES matter. I asked about this previously in this question. @Xi'an commented (4th comment): "inference about which model is correct does not depend on the pseudo-priors".
Recently I read comments from Martyn Plummer that contradict my understanding of Carlin & Chib. Martyn says: "In order for the Carlin-Chib method to work, the pseudo-prior must match the posterior when the model is true."
(I am NOT saying that Plummer contradicts Carlin & Chib; only that he contradicts my understanding of Carlin & Chib's claim).
All of this leaves me with five questions:
- What is going on here? Provided that the model converges and yields a good effective sample size from the posterior, will my inference about which variables to include in a model depend on my pseudo-prior?
- If not, how do we square this with my intuition and Plummer's comment? If so, how do we square this with Carlin & Chib's paper and Xi'an's comment (4th comment)?
- If my understanding of Plummer's comment is correct, and the pseudo-priors must correspond to the posterior when the variable is included... does this mean it's impermissible to pseudo-priors corresponding exactly to the true priors? This would mean that pseudo-priors are much more than a convenient technique to improve the mixing in the MCMC!!
What if the indicator variable turns on and off a part of the model with several parameters (for example, a random effect with a grand mean, a variance, and n group effects)? Which of the following are permissible (in order of how confident I am that the approach is permissible)? Is there a better approach that I do not list?
i. Use a pseudo-prior that approximates the full joint posterior distribution of all of the parameters.
ii. If mixing is acceptably non-atrocious, don't use pseudo-priors at all (i.e. use pseudo-priors equivalent to the true priors).
iii. Use a pseudo-prior based on the univariate posterior distributions for each parameter, but don't worry about how they are jointly distributed.
iv. Following the apparently plain language of Carlin & Chib, use any pseudo-prior that gives computationally efficient mixing in the MCMC chains, as "the form of [the pseudo-prior] is irrelevant".
What does @Xi'an mean in the first comment on my question in saying "the pseudo-priors need correction in an importance sampling type of correction."