# Can we apply BIC to select between models in which the parameters are estimated by MCMC

I never considered this problem before. Assume we have two models, $A$ and $B$, we estimate the parameter in $A$ and $B$ respectively by sample mean. Two questions:

1. We can always use the means of both samples to estimate the parameters, then is it possible to do so via the mode of the sample?
2. Once we get estimates of the parameters, can we use BIC to select models? As the definition of BIC involves the maximized likelihood function, $\hat{L}$, then it seems that BIC is not suitable as a selection criterion in MCMC, is it true?

If you operate a Bayesian analysis by running an MCMC algorithm, it means you obtain a sample of $\theta_A$'s and a sample of $\theta_B$'s, which are representative of the posterior distributions. From those samples, you can derive Bayes estimates according to whatever loss or criterion you choose, like posterior means, posterior median or posterior mode (which does not correspond to a loss function). The latter is provided by$$\hat\theta_A^\text{MAP}=\arg\max\pi_A(\theta_A)f_A(x|\theta_A)$$that I assume you can compute.
BIC is a pseudo-Bayesian criterion that does not account for the prior distribution but instead only uses the likelihood function and an estimator. You can substitute the MAP estimate for the MLE and keep the same asymptotic validity for the BIC convergence. Of course, once you have produce an MCMC sample it would be more natural to evaluate the evidence$$\int_{\Theta_A} \pi_A(\theta_A)f_A(x|\theta_A)\text{d}\theta_A$$than the approximate BIC.
• I think it is possible that the MCMC methods have failed to find the true MLE for each of the models as we just have the posteriori samples and we can not includes are the possible value of the $\theta$, is it true? I think the evidence is fine to compare the performance of models but it doesn't consider the impact of the number of free parameters, are there any more criterions that can involve this function in MCMC? Commented Nov 23, 2015 at 18:49