I am following Pattern recognition and machine learning by Byshop and I was trying to derive myself the predictive distribution resulting from a new data point. For the sake of clarity I post the piece of text in question: enter image description here

where $w$ are the parameters of the model treated as a random variable since we are in the Bayesian setting.

Anyhow I was trying to derive this myself and what I would do is

$$p(t|x, \textbf{x}, \textbf{t}) = \int p(t|x, \textbf{x}, \textbf{t},w) dw = \int \frac{p(t,x, \textbf{x}, \textbf{t},w)}{p(x, \textbf{x}, \textbf{t},w)} dw = \dots$$

And then I am unsure how to proceed. How could this be done?

EDIT: I was trying to apply Bayes rule with multiple conditions as shown here but to no avail.


The problem is your first step. You need to marginalize out the "w" term, so you should be integrating the joint distribution of t and w over all w. It should be:

$p(t|x, \textbf{x}, \textbf{t}) = \int p(t, w|x, \textbf{x}, \textbf{t}) dw$

Then, you can use the conditional probability rule,

$\int p(t, w|x, \textbf{x}, \textbf{t}) dw = \int p(t|w, x, \textbf{x}, \textbf{t})p(w|x,\textbf{x},\textbf{t})dw$

Now, recognizing that $t$ does not depend on $\textbf{x}$ and $\textbf{t}$, and $w$ does not depend on $x$,

$\int p(t|w, x, \textbf{x}, \textbf{t})p(w|x,\textbf{x},\textbf{t})dw = \int p(t|w,x)p(w| \textbf{x}, \textbf{t})dw$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.