# Are there two motivations for Bayesian information criteria?

Are there two motivations for all these Bayesian information criteria? I am only aware of the motivation of "expected out-of-sample prediction score."

Let the in-sample data be $$y$$ and the parameter be called $$\theta$$. Assume that $$y\mid \theta \sim p(y \mid \theta)$$ and that the prior is $$p(\theta)$$. Call the posterior mean $$\hat{\theta}(y)$$, and let the out-of-sample data be called $$Y$$. This closely follows the notation found here.

The "effective number of parameters" described by that paper is \begin{align*} p_D &= E_{\theta \mid y}\left[ - 2 \log p(y \mid \theta) \right] + 2 \log p(y \mid \hat{\theta}(y)) \\ \end{align*}

Why isn't there being an expectation taken with respect to unobserved data $$Y$$ in the above expression? I thought the whole point of this class of model selection strategies was to approximate (ideally) $$E_Y\left[ E_{\theta \mid y}\left[ \log p(Y \mid \theta) \right] \right]$$, or more realistically \begin{align*} E_Y\left[-2 \log p(Y \mid \hat{\theta}(y)) \right] &= - 2\log p(y \mid \hat{\theta}(y)) \\ &+ \underbrace{E_Y\left[ -2\log p(Y \mid \hat{\theta}(y)) \right] + 2\log p(y \mid \hat{\theta}(y))}_{\text{a better }p_D \text{?}} \end{align*}

But clearly $$E_Y\left[ -2\log p(Y \mid \hat{\theta}(y)) \right] \neq E_{\theta \mid y}\left[ - 2 \log p(y \mid \theta) \right].$$ Is that just a commonly-used approximation?

• You could consider the possibility that there are more than two. – Glen_b Jan 13 at 6:53
• Just avoid DIC, it is a toxic measure! – Xi'an Jan 13 at 8:42