# Why does the posterior predictive distribution involve an integral?

Given the posterior predictive distribution for a new data point $$x^*$$, the posterior predictive distribtion given some data $$(X,Y)$$

\begin{align*} p(y^*|x^*,X,Y) = \int p(y^*|x^*,\omega) p(\omega|X,Y) \, d \omega \end{align*}

gives us the distribution of future predicted data $$y^*$$.

What is the logic behind the integral? Why do we need an integral here?

Cheers

EDIT: What is the intuition behind the integral? Obviously it measures some kind of area within $$p(y^*|x^*,\omega) p(\omega|X,Y)$$, which are nothing more than mass functions.

• Perhaps $\omega$ is unobserved so the best you can do is integrate it out using a distribution $p(\omega \lvert X,Y)$ conditioned on the variables you observe $X$,$Y$. But unless you provide more detail on the set up, the is simply guessing. – Jesper for President May 27 '20 at 8:14
• @JesperforPresident thanks for your comment. The new data point $x^*$ depends on parameter $\omega$, whereas the posterior distribution of $\omega$ depends on given data $X$. The definition of the posterior predictive distribution I used is from wikipedia. Perhaps the intuition of using an integral here isn't clear to me. – MJimitater May 27 '20 at 8:21

You don't exactly know $$\omega$$ but you have some idea, a distribution based on the previous data you've seen, which is described by $$p(\omega|X,Y)$$. If you had a constant $$\omega_0$$, the posterior predictive distribution would be $$p(y^*|x^*,\omega_0)$$, but the integral is basically an expected value (i.e. a weighted average) over all possible $$\omega$$.
By the way, the integral is at the same time comes from total probability law: $$p(y^*|x^*,X,Y)=\int \underbrace{p(y^*|x^*,\omega,X,Y)p(\omega|X,Y)}_{p(y,\omega|x^*,X,Y)}d\omega=\int p(y^*|x^*,\omega)p(\omega|X,Y)d\omega$$
The first term inside the integral is simplified as $$p(y^*|x^*,\omega,X,Y)=p(y^*|x^*,\omega)$$, because when you actually know the model parameters, you don't need the training data to learn them. So, given the input $$x^*$$, the output $$y^*$$ is assumed to be dependent on only the model parameters, $$\omega$$.