In the book "Pattern recognition and machine learning" by Christopher M. Bishop, at page 374

The joint distribution corresponding to this graph is again obtained from our general formula (8.5) to give

$$ p(a,b,c) = p(a) p(c|a) p(b|c) $$ First of all, suppose that none of the variables are observed. Again, we can test to see if $a$ and $b$ are independent by marginalizing over $c$ to give

$$ p(a,b) = p(a) \sum\limits_c p(c|a) p(b|c) = p(a) p(b|a)$$

I did not understand this last passage. How does it go from $$ p(a) \sum\limits_c p(c|a) p(b|c) $$ to $$ p(a) p(b|a)$$?

I have tried to explicit the conditional distribution:

$$ \sum\limits_c p(c|a) p(b|c) = \sum\limits_c \dfrac{p(c,a)}{p(a)} \dfrac{p(b,c)}{p(c)} $$

I have also tried to use the Bayes theorem

$$ \sum\limits_c p(c|a) p(b|c) = \sum\limits_c p(a|c) \dfrac{p(c)}{p(a)} p(b|c) $$

but nothing


The second formula derives directly from the first:

$$ p(a,b,c) = p(a) p(c|a) p(b|c) $$

Indeed $$ p(a,b) =\sum\limits_c p(a,b,c) $$

$=\sum\limits_c p(a) p(c|a) p(b|c) $ (according to the 1st statement) $$= p(a) \sum\limits_c p(c|a) p(b|c) $$

Beside you know that (Bayes'rule): $$ p(a,b) = p(a) p(b|a)$$

Which leads to

$$ p(a,b) = p(a) \sum\limits_c p(c|a) p(b|c) = p(a) p(b|a)$$

EDIT: it can be useful to provide here the link toward the material: pattern recognition and machine learning

Looking at the corresponding page provides some useful insights on the context and it can be relevant to note that the given formula is not true in the general case but corresponds here to a specific probability graph configuration.

A more general formula being for instance:

$$ p(a,b,c) = p(a) p(b|a) p(c|a,b)$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.