Bayes with Multiple Conditions and Priors

Question

Basic question:

This should be a textbook application of Bayes rule to multiple conditions, but I am having trouble seeing where the prior comes in and why there's an integral in the final probability.

I refer to the following two lecture slides:

The correct answer is 3 - I would be really grateful if anybody could show the derivation.

Answer 1

I will provide a partial answer because I'm convinced this is self-study, thus according to the rules for that kind of questions, I cannot give a complete answer.

From the definition of joint distribution between the variables $y_*$ and $f$, conditional on $\mathcal{D}$ and $x_*$:

$p(y_*|\mathcal{D},x_*)=\int p(y_*,f|\mathcal{D},x_*)df$

Then, from the definition of conditional density:

$\int p(y_*,f|\mathcal{D},x_*)df=\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D},x_*)df$

Now, the only way I can imagine your equation 3 can be valid, is if $f$ is independent of $x_*$ conditionally on $\mathcal{D}$. Otherwise, I'm convinced the equation is false. Assuming this conditional independence, then

$\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D},x_*)df=\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D})df$

To summarize, we proved that

$p(y_*|\mathcal{D},x_*)=\int p(y_*|f,\mathcal{D},x_*)p(f|\mathcal{D})df$

Now, you can easily get the desired result if you use the Bayes' rule to express the posterior probability $p(f|\mathcal{D})$ as a function of likelihood, prior and a normalizing constant.