0
$\begingroup$

Are there two motivations for all these Bayesian information criteria? I am only aware of the motivation of "expected out-of-sample prediction score."

Let the in-sample data be $y$ and the parameter be called $\theta$. Assume that $y\mid \theta \sim p(y \mid \theta)$ and that the prior is $p(\theta)$. Call the posterior mean $\hat{\theta}(y)$, and let the out-of-sample data be called $Y$. This closely follows the notation found here.

The "effective number of parameters" described by that paper is \begin{align*} p_D &= E_{\theta \mid y}\left[ - 2 \log p(y \mid \theta) \right] + 2 \log p(y \mid \hat{\theta}(y)) \\ \end{align*}

Why isn't there being an expectation taken with respect to unobserved data $Y$ in the above expression? I thought the whole point of this class of model selection strategies was to approximate (ideally) $E_Y\left[ E_{\theta \mid y}\left[ \log p(Y \mid \theta) \right] \right]$, or more realistically \begin{align*} E_Y\left[-2 \log p(Y \mid \hat{\theta}(y)) \right] &= - 2\log p(y \mid \hat{\theta}(y)) \\ &+ \underbrace{E_Y\left[ -2\log p(Y \mid \hat{\theta}(y)) \right] + 2\log p(y \mid \hat{\theta}(y))}_{\text{a better }p_D \text{?}} \end{align*}

But clearly $$ E_Y\left[ -2\log p(Y \mid \hat{\theta}(y)) \right] \neq E_{\theta \mid y}\left[ - 2 \log p(y \mid \theta) \right]. $$ Is that just a commonly-used approximation?

$\endgroup$
  • $\begingroup$ You could consider the possibility that there are more than two. $\endgroup$ – Glen_b Jan 13 at 6:53
  • $\begingroup$ Just avoid DIC, it is a toxic measure! $\endgroup$ – Xi'an Jan 13 at 8:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.