# Posterior Predictive Distribution as Expectation of Likelihood

Say we have a posterior predictive density:

$$p(\tilde{y}|\mathbf{y}) = \int p(\tilde{y}|\theta)p(\theta|\mathbf{y})d\theta$$

In Hoff's Bayesian Statistical Methods text, he suggests that to obtain an approximation of $$p(\tilde{y}|\mathbf{y})$$ by sampling from posterior distribution, and computing $$\frac{1}{S}\sum_{s=1}^Sp(\tilde{y}|\theta^{(s)})$$.

He justifies this by stating $$p(\tilde{y} | \mathbf{y})$$ is the posterior expectation of $$p(\tilde{y}|\theta)$$, but I actually can't see the equivalence. How does one derive $$p(\tilde{y} | \mathbf{y})$$ from $$p(\tilde{y}|\theta)$$?

## 1 Answer

$$\newcommand{\y}{\mathbf y}$$We have $$E_{\theta|\y}\left[f(\theta)\right] = \int f(\theta) p(\theta | \y)\,\text d\theta$$ just by definition of expectation (and you could cite the LOTUS as well), and since $$p(\theta|\y)$$ is the posterior density this is the posterior expectation of $$f(\theta)$$. Now choose $$f(\theta) = p(\tilde y | \theta)$$ and then $$E_{\theta|\y}\left[p(\tilde y | \theta)\right] = \int p(\tilde y | \theta) p(\theta | \y)\,\text d\theta.$$

I'm not sure if you also are wondering about the justification of this integral in the first place, but typically the data are assumed independent given the generating parameters so for a new point $$\tilde y$$ you'd have $$\tilde y \perp \y | \theta$$ which means $$p(\tilde y | \y) = \int p(\tilde y , \theta | \y)\,\text d\theta \\ = \int p(\tilde y | \theta , \y) p(\theta | \y)\,\text d\theta \\ = \int p(\tilde y | \theta) p(\theta | \y)\,\text d\theta \\ = E_{\theta|\y}\left[p(\tilde y | \theta)\right]$$

so you can use the law of large numbers with posterior samples to produce an estimator of this density.

• To compute $p(\tilde{y}|y)$, first sample from $\tilde{y}$ from the data distribution and then for this $\tilde{y}$, continuously sample $\theta^{(i)}$ from the posterior distribution and compute $p(\tilde{y}|\theta)$ ? how does the expectation scalar quantity be turned to a probability distribution for $p(\tilde{y}|y)$ ? – calveeen Aug 10 '20 at 13:43