0
$\begingroup$

I'm reading Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville. In chapter 3 about graphical models, to reduce the model complexity, we assume that certain conditional independence assumptions hold. Specifically, in equation 3.52 on page 75 (I quote)

... suppose we have three random variables: $a$, $b$ and $c$. Suppose that $a$ influences the value of $b$, and $b$ influences the value of $c$ , but that $a$ and $c$ are independent given $b$. We can represent the probability distribution over all three variables as a product of probability distributions over two variables:

$p(a,b,c) = p(a)p(b|a)p(c|b)$

I understand the last statement "$a$ and $c$ are independent given $b$" as conditional independence, and I would write it down as,

$p(a,c|b) = p(a|b)p(c|b)$

Is this right ? How should I prove the claim mentioned in the book ? I tried to use my result (of conditional independence) with the chain rule or product rule of conditional probabilities, but I could not arrive the claim (3.52 on page 75) mentioned in the book.

$\endgroup$
1
$\begingroup$

You're right about $𝑝(𝑎,𝑐|𝑏)=𝑝(𝑎|𝑏)𝑝(𝑐|𝑏)$, it is the definition of conditional independence. For the formula, without any information about the variables/events, we normally have the following, holding always: $$p(a,b,c)=\underbrace{p(a)p(b|a)}_{p(a,b)}p(c|a,b)$$

When, $c$ and $a$ are independent given $b$, we also have $p(c|a,b)=p(c|b)$ as in the formula. This is very intuitive because $b$ is given, and having $a$ additionally doesn't influence the distribution of $c$, since they're defined as conditionally independent.

We can also deduce this relationship using your formula: $$p(c|a,b)=\frac{p(a,b,c)}{p(a,b)}=\frac{p(a,c|b)p(b)}{p(a|b)p(b)}=\frac{p(a|b)p(c|b)}{p(a|b)}=p(c|b)$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.