6
$\begingroup$

Given the posterior predictive distribution for a new data point $x^*$, the posterior predictive distribtion given some data $(X,Y)$

\begin{align*} p(y^*|x^*,X,Y) = \int p(y^*|x^*,\omega) p(\omega|X,Y) \, d \omega \end{align*}

gives us the distribution of future predicted data $y^*$.

What is the logic behind the integral? Why do we need an integral here?

Cheers

EDIT: What is the intuition behind the integral? Obviously it measures some kind of area within $p(y^*|x^*,\omega) p(\omega|X,Y)$, which are nothing more than mass functions.

$\endgroup$
2
  • 1
    $\begingroup$ Perhaps $\omega$ is unobserved so the best you can do is integrate it out using a distribution $p(\omega \lvert X,Y)$ conditioned on the variables you observe $X$,$Y$. But unless you provide more detail on the set up, the is simply guessing. $\endgroup$ – Jesper for President May 27 '20 at 8:14
  • $\begingroup$ @JesperforPresident thanks for your comment. The new data point $x^*$ depends on parameter $\omega$, whereas the posterior distribution of $\omega$ depends on given data $X$. The definition of the posterior predictive distribution I used is from wikipedia. Perhaps the intuition of using an integral here isn't clear to me. $\endgroup$ – MJimitater May 27 '20 at 8:21
5
$\begingroup$

You don't exactly know $\omega$ but you have some idea, a distribution based on the previous data you've seen, which is described by $p(\omega|X,Y)$. If you had a constant $\omega_0$, the posterior predictive distribution would be $p(y^*|x^*,\omega_0)$, but the integral is basically an expected value (i.e. a weighted average) over all possible $\omega$.

By the way, the integral is at the same time comes from total probability law: $$p(y^*|x^*,X,Y)=\int \underbrace{p(y^*|x^*,\omega,X,Y)p(\omega|X,Y)}_{p(y,\omega|x^*,X,Y)}d\omega=\int p(y^*|x^*,\omega)p(\omega|X,Y)d\omega$$

The first term inside the integral is simplified as $p(y^*|x^*,\omega,X,Y)=p(y^*|x^*,\omega)$, because when you actually know the model parameters, you don't need the training data to learn them. So, given the input $x^*$, the output $y^*$ is assumed to be dependent on only the model parameters, $\omega$.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.