3
$\begingroup$

I am trying to obtain the right hand side of equation 1.68 in Pattern Recognition and Machine Learning by Bishop. I was treating the problem as having four random variables $x,t,D,w$ where $D=(X,T)$ then I only obtain this:

$$P(t,x,D)=\int P(t,x,D,w)dw$$ $$P(t|x,D)P(x,D)=\int P(t|x,D,w)P(x,D,w)dw$$ $$P(t|x,D)=\int P(t|x,D,w)P(w|x,D)dw$$

Could anybody help me please?

$\endgroup$
2
  • $\begingroup$ Please use math typesetting. math.meta.stackexchange.com/questions/5020/… $\endgroup$
    – Sycorax
    Sep 26 '17 at 20:53
  • 1
    $\begingroup$ We're not copyright police, but a pdf of a whole textbook seems a bit much to swallow. I've changed the link. If that makes your question harder to follow for someone without the text in front of them, note that questions are supposed to be self-contained anyway - to at least make sense without reference to external sources. Quote relevant parts if you need to. $\endgroup$ Sep 27 '17 at 8:24
3
$\begingroup$

The book sneakily invoked the concept of "conditional independence".

Suppose we have variables $A,$ $B,$ and $C,$ and that $A$ and $B$ are conditionally independent given $C.$ This means that $P(A \mid B, C) = P(A \mid C).$ That is, if $C$ is observed, then $A$ is independent of $B.$ However, that independence is conditional, so it's still true that $P(A \mid B) \ne P(A)$ in general.

In this case, $t$ is conditionally independent of $D$ given $w.$ The reason for this is that $t$ solely depends on $w$ and $x,$ but if you don't know $w$ then $D$ gives you a hint to the value of $w.$ However, if you do know $w$ then $D$ is no longer useful for determining the value of $t.$ This explains why $D$ was omitted from $P(t \mid x, w, D)$ but not from $P(t \mid x, D).$

Similarly, $w$ is entirely independent of $x$ so $P(w \mid x, D) = P(w \mid D).$

$\endgroup$
2
  • $\begingroup$ Now it makes sense, thanks! so just to confirm, is my derivation correct? $\endgroup$ Sep 26 '17 at 20:29
  • $\begingroup$ Yes, it looks right to me. $\endgroup$ Sep 26 '17 at 22:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.