# How are the product/sum rules of probability used here? (Bishop's Pattern Recognition)

For a grad school course I'm studying Bishop's Pattern Recognition and I can't follow how he derives the following formula.

I've tried applying the sum rule first and then the product rule and vice versa but nothing lead me to the correct equation. Can anyone help out? Thanks.

The trick here is that the prediction $t$ is conditionally independent of the data $\mathbf{x}, \mathbf{t}$ given the model parameters $w$: $$t \perp\!\!\!\perp \mathbf{x}, \mathbf{t} \,\,\vert\,\, w$$ That means: If we know the model weights, knowing the training data will not alter the model prediction.

Another independence holds for the second factor—weights $w$ are independent on the current sample $x$, they only depend on the training data $\mathbf{x}, \mathbf{t}$.

The rest is simple:

\begin{align} p(t|x,\mathbf{x},\mathbf{t})&=\int p(t,w|x,\mathbf{x},\mathbf{t})\,\mathrm{d}w \\ &=\int p(t|x,w,\mathbf{x},\mathbf{t})p(w|x,\mathbf{x},\mathbf{t})\,\mathrm{d}w \\ &=\int p(t|x,w)p(w|\mathbf{x},\mathbf{t}) \,\mathrm{d}w \end{align}

1. Sum rule
2. Chain rule
3. Independence
• Thanks Jan! It makes sense to me, but would never have figured it out myself. – David Sep 6 '18 at 10:04
• If you're stuck, it usually helps writing explicitly the formulas hidden behind $p$, e.g. $p(t|x,w,\mathbf{x},\mathbf{t}) = \mathcal{N}(t|y(x,w), \beta)$ (1.60), now you can immediately see that there is no $\mathbf{x},\mathbf{t}$ on the right side. – Jan Kukacka Sep 6 '18 at 10:13