1
$\begingroup$

This is actually a probability marginalization question that I encountered in graphic models section of PRML by Bishop (question about equation 8.26 page 391). Assume I have the following graphic model

enter image description here

Therefore the joint probability density of the variables will factor to $p(a,b,c)=p(a)p(c|a)p(b|c)$. Now assume I want to marginalize over $c$, the book says

$$\sum_c p(a,b,c) = \sum_c p(a)p(c|a)p(b|c) = p(a) \sum_c p(c|a)p(b|c)= p(a)p(b|a)$$

This means that $p(b|a) = \sum_c p(c|a)p(b|c)$, how to prove this?! If it was like this $\sum_c p(c|a)p(b|c,a)$ then one can reason that $\sum_c p(c|a)p(b|c,a) = \sum_c \frac{p(a,c)}{p(a)}\frac{p(a,b,c)}{p(a,c)} = \sum_c p(b,c|a)=p(b|a)$. But I can't conclude the same result with $\sum_c p(c|a)p(b|c)$. What am I getting wrong?

Thanks in advance

$\endgroup$
3
$\begingroup$

You need to use the fact that $p(b|c,a)=p(b|c)$ i.e. $a$ and $b$ conditionally independent given $c$. This is directly visible on the graph and also proved with:

$p(b|c,a)=\frac{p(b,c,a)}{p(c,a)}=\frac{p(a)p(c|a)p(b|c)}{\sum_{b}p(a)p(c|a)p(b|c)}=\frac{p(b|c)}{\sum_{b}p(b|c)}=p(b|c).$

Use this additional equality at the beginning of your development to get the result.

$\endgroup$
1
  • $\begingroup$ Thank you for your answer, I think you should replace $c$ with $a$ in your answer and it will match the notation in my question but overall nice idea for proof. $\endgroup$ – K.K.McDonald Jul 27 '20 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.