Are pseudopriors required in Bayesian model selection with hierarchical models?

Question

Say I have a set of $K$ models and I want to perform Bayesian model selection to see which one of those best describes my data. So I add a categorical variable with $K$ different values that indicates which model is currently tested, and I only estimate the parameters of the one model that is currently selected (Bayesian model selection).

However, as far as I understand this approach, doing this naively can create "funnels" in the probability distribution, because at any time only the parameters of a single model are constrained by the data, whereas the others can wander off freely. So the usual approach is to add pseudopriors, based on posteriors from previous runs of each model, which constrain the models currently not selected.

Now, what if I also have data from $N$ different groups (e.g. participants), so that I have a hierarchical structure. I have reasons to assume that different models best describe the data from different groups. How does this interplay with Bayesian model selection? There are some steps that are clear to me, and some that are not.

1. The model now needs $N$ different categorical variables. One for each group in the grouping structure. These can be sampled from the same multinomial distribution (with hyperpriors, so we employ the grouping structure).
2. Whenever a model is selected for one of the groups, parameters for that group are sampled from the hyperprior for that model.

But what about, when a model is currently not selected for one of the groups? I know I could just add pseudopriors for each model and group combination, but then I would have to estimate $N\times K$ posteriors first and I would have to include $N\times K$ pseudopriors in the final model selection step.

If I understood the approach with pseudopriors correctly, then I have the feeling that I don't really need pseudopriors in this case. But I cannot really justify this. Pseudopriors are meant to constrain the parameters when the model is not currently sampled. However, the hyperprior also constrain the parameters, so in the hierarchical setting they cannot wander off anyway. So wouldn't it be enough to just sample from the hyperpriors whenever a model is currently not selected?

Answer 1

From a theoretical perspective, the Bayesian comparison of $M$ models $\mathfrak M_m$ $m=1,...,M$ proceeds by the comparison of their posterior probabilities $$\pi(M_m|\mathbf x) \propto \pi(M_m) \int_{\Theta_m} f_m(\mathbf x|\theta) \pi_m(\theta_m)\,\text d\theta_m$$and therefore implies all priors over all models and over all model parameters. The selection of a model cannot operate from a single model perspective. Approaches based on pseudo-priors are only useful from a computational perspective, as in Carlin and Chib (1995), when one is exploring the posterior over the product space $\prod \Theta_m$ by Monte Carlo or Markov chain Monte Carlo methods.