Basic question:

This should be a textbook application of Bayes rule to multiple conditions, but I am having trouble seeing where the prior comes in and why there's an integral in the final probability.

I refer to the following two lecture slides:

Lecture Slide 1

Lecture Slide 2 (Correct answer is 3)

The correct answer is 3 - I would be really grateful if anybody could show the derivation.

  • $\begingroup$ Ps added the required self-study tag. $\endgroup$
    – DeltaIV
    Feb 15, 2017 at 7:10

1 Answer 1


I will provide a partial answer because I'm convinced this is self-study, thus according to the rules for that kind of questions, I cannot give a complete answer.

From the definition of joint distribution between the variables $y_*$ and $f$, conditional on $\mathcal{D}$ and $x_*$:

$p(y_*|\mathcal{D},x_*)=\int p(y_*,f|\mathcal{D},x_*)df$

Then, from the definition of conditional density:

$\int p(y_*,f|\mathcal{D},x_*)df=\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D},x_*)df$

Now, the only way I can imagine your equation 3 can be valid, is if $f$ is independent of $x_*$ conditionally on $\mathcal{D}$. Otherwise, I'm convinced the equation is false. Assuming this conditional independence, then

$\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D},x_*)df=\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D})df$

To summarize, we proved that

$p(y_*|\mathcal{D},x_*)=\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D})df$

Now, you can easily get the desired result if you use the Bayes' rule to express the posterior probability $p(f|\mathcal{D})$ as a function of likelihood, prior and a normalizing constant.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.