Can we apply BIC to select between models in which the parameters are estimated by MCMC

Question

I never considered this problem before. Assume we have two models, $A$ and $B$, we estimate the parameter in $A$ and $B$ respectively by sample mean. Two questions:

1. We can always use the means of both samples to estimate the parameters, then is it possible to do so via the mode of the sample?
2. Once we get estimates of the parameters, can we use BIC to select models? As the definition of BIC involves the maximized likelihood function, $\hat{L}$, then it seems that BIC is not suitable as a selection criterion in MCMC, is it true?

Answer 1

A preliminary remark is that MCMC has nothing to do with problems of inference, it is a Monte Carlo algorithm that produces simulations from a given probability distribution.

If you operate a Bayesian analysis by running an MCMC algorithm, it means you obtain a sample of $\theta_A$'s and a sample of $\theta_B$'s, which are representative of the posterior distributions. From those samples, you can derive Bayes estimates according to whatever loss or criterion you choose, like posterior means, posterior median or posterior mode (which does not correspond to a loss function). The latter is provided by$$\hat\theta_A^\text{MAP}=\arg\max\pi_A(\theta_A)f_A(x|\theta_A)$$that I assume you can compute.

BIC is a pseudo-Bayesian criterion that does not account for the prior distribution but instead only uses the likelihood function and an estimator. You can substitute the MAP estimate for the MLE and keep the same asymptotic validity for the BIC convergence. Of course, once you have produce an MCMC sample it would be more natural to evaluate the evidence$$\int_{\Theta_A} \pi_A(\theta_A)f_A(x|\theta_A)\text{d}\theta_A$$than the approximate BIC.