For a grad school course I'm studying Bishop's Pattern Recognition and I can't follow how he derives the following formula.

enter image description here

I've tried applying the sum rule first and then the product rule and vice versa but nothing lead me to the correct equation. Can anyone help out? Thanks.


The trick here is that the prediction $t$ is conditionally independent of the data $\mathbf{x}, \mathbf{t}$ given the model parameters $w$: $$ t \perp\!\!\!\perp \mathbf{x}, \mathbf{t} \,\,\vert\,\, w $$ That means: If we know the model weights, knowing the training data will not alter the model prediction.

Another independence holds for the second factor—weights $w$ are independent on the current sample $x$, they only depend on the training data $\mathbf{x}, \mathbf{t}$.

The rest is simple:

$$ \begin{align} p(t|x,\mathbf{x},\mathbf{t})&=\int p(t,w|x,\mathbf{x},\mathbf{t})\,\mathrm{d}w \\ &=\int p(t|x,w,\mathbf{x},\mathbf{t})p(w|x,\mathbf{x},\mathbf{t})\,\mathrm{d}w \\ &=\int p(t|x,w)p(w|\mathbf{x},\mathbf{t}) \,\mathrm{d}w \end{align} $$

  1. Sum rule
  2. Chain rule
  3. Independence
  • $\begingroup$ Thanks Jan! It makes sense to me, but would never have figured it out myself. $\endgroup$ – David Sep 6 '18 at 10:04
  • $\begingroup$ If you're stuck, it usually helps writing explicitly the formulas hidden behind $p$, e.g. $p(t|x,w,\mathbf{x},\mathbf{t}) = \mathcal{N}(t|y(x,w), \beta)$ (1.60), now you can immediately see that there is no $\mathbf{x},\mathbf{t}$ on the right side. $\endgroup$ – Jan Kukacka Sep 6 '18 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.