# What is the common criterion to decide the performance of prior selection in MCMC

For a model with likelihood $p(Y|\theta)$, in which $Y$ is the data and $\theta$ is the parameters. Based on Bayes Rule, we have the posterior

$p(\theta|Y) \propto p(Y|\theta) p(\theta)$

My question is as follows: if we have different prior distributions on $\theta$, e.g., $p(\theta)$ can be a non-informative prior $p(\theta)\propto 1$ or a Gaussian prior $p(\theta) \propto \exp(-\frac{\theta^2}{2})$, if we run a random walk Metropolis Hasting algorithm aiming at posteriors based on different priors, we have one chain for each, then how can we compare the influence of these priors on the posteriors, when considering the MCMC samples?

## 1 Answer

Bayesian theory suggests that you compute a Bayes factor comparing both [Bayesian] models through the marginal likelihoods of the data $$\mathfrak{B}_{12}(x)=\dfrac{\int f(x|\theta)\,\pi_1(\theta)\,\text{d}\theta}{\int f(x|\theta)\,\pi_2(\theta)\,\text{d}\theta}$$ This may actually be one of the most legitimate usages of the Bayes factor since the priors are then the object of interest, rather than an entry difficult to calibrate.

There are two caveats though:

1. the Bayes factor is not properly defined when one or both of the priors are improper (this is connected with the Jeffreys-Lindley paradox);
2. the computation of the Bayes factor is delicate, although this may be the most favourable setting since you can use bridge sampling there. Along with many other methods.
• Thanks @Xi'an firstly. It is reasonable to accept the Bayes factor, but my understanding is that Bayes factor concentrates on selection of model. Might be my quiz is not clear, actually what I mean is that influence of prior on the efficiency of sampling in MCMC. I read the paper by Andrew Gelman, link, he introduced calibration in that paper, which I am not quit understand, then I would like to know if other measure of efficiency, for example, IACT, can be used in analysis the performance of prior. – Fly_back Sep 17 '16 at 23:21
• @Fly_back: Given that the targets associated with two different priors are different, it seems hard to make a comparison between the MCMC samplers. Unless you use one as a proposal for the other and compare the effective sample sizes maybe... – Xi'an Sep 18 '16 at 8:41