0
$\begingroup$

Are there two motivations for all these Bayesian information criteria? I am only aware of the motivation of "expected out-of-sample prediction score."

Let the in-sample data be $y$ and the parameter be called $\theta$. Assume that $y\mid \theta \sim p(y \mid \theta)$ and that the prior is $p(\theta)$. Call the posterior mean $\hat{\theta}(y)$, and let the out-of-sample data be called $Y$. This closely follows the notation found here.

The "effective number of parameters" described by that paper is \begin{align*} p_D &= E_{\theta \mid y}\left[ - 2 \log p(y \mid \theta) \right] + 2 \log p(y \mid \hat{\theta}(y)) \\ \end{align*}

Why isn't there being an expectation taken with respect to unobserved data $Y$ in the above expression? I thought the whole point of this class of model selection strategies was to approximate (ideally) $E_Y\left[ E_{\theta \mid y}\left[ \log p(Y \mid \theta) \right] \right]$, or more realistically \begin{align*} E_Y\left[-2 \log p(Y \mid \hat{\theta}(y)) \right] &= - 2\log p(y \mid \hat{\theta}(y)) \\ &+ \underbrace{E_Y\left[ -2\log p(Y \mid \hat{\theta}(y)) \right] + 2\log p(y \mid \hat{\theta}(y))}_{\text{a better }p_D \text{?}} \end{align*}

But clearly $$ E_Y\left[ -2\log p(Y \mid \hat{\theta}(y)) \right] \neq E_{\theta \mid y}\left[ - 2 \log p(y \mid \theta) \right]. $$ Is that just a commonly-used approximation?

$\endgroup$
  • $\begingroup$ You could consider the possibility that there are more than two. $\endgroup$ – Glen_b Jan 13 at 6:53
  • $\begingroup$ Just avoid DIC, it is a toxic measure! $\endgroup$ – Xi'an Jan 13 at 8:42

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.