1
$\begingroup$

Sorry if this is a silly question but it comes from reading a book on directed graphical models. They show algebraically that two variables $x$, $z$ are conditionally independent given $y$.

It says: "first consider a chain structure: x -> y -> z (here y is observed), then:

$$p(x, z | y) = {p(z|y)p(y|x)p(x)\over{p(y)}} = {p(x, y)p(z|y)\over{p(y)}} = p(x|y)p(z|y)$$

I've tried to go over this step by step but only only way I can arrive at the conclusion given in the book is to first assume that $x$ and $z$ are conditionally independent given $y$, and that simply seems circular. This is what I do

If they are conditionally independent then

$$p(x, z|y) = p(x|y)p(z|y)$$

apply bayes rule to $p(x|y)$:

$$= {p(y|x)p(x)p(z|y)\over{p(y)}}$$

then $p(y|x)p(x)$ can be written as $p(x, y)$, and expand again to make it $p(x|y)p(y)$:

$$=p(x|y)p(z|y)$$

isn't this just wrong because it's circular?

My other thought was that Bayes rule was applied at the start:

$$p(x,z|y) = {p(y|x,z)p(x,z)\over{p(y)}}$$

but now I can't work out how $p(y|x,z)p(x,z)$ is equal to $p(z|y)p(y|x)p(x)$

$\endgroup$
  • 2
    $\begingroup$ It's not true in general that $p(x,z \mid y) = p(x \mid y) p(z \mid y)$. So, the book was probably talking about some specific set of circumstances (and the proof would follow from there). Can you provide a little more context about what the book was saying? $\endgroup$ – user20160 Mar 6 at 16:23
  • $\begingroup$ the other context I can add (edited the post to add it) is that it says consider a chain structure x -> y -> z, and that y is observed. in which case I guess I'm not sure why adding that information would change things $\endgroup$ – atomsmasher Mar 6 at 16:34
1
$\begingroup$

What the book is showing is that the specified graph (i.e. chain structure in this case) implies conditional independence.

The chain structure $X \to Y \to Z$ indicates that the joint distribution factorizes as:

$$p(x,y,z) = p(z \mid y) p(y \mid x) p(x)$$

Using the definition of conditional probability, the conditional distribution of $X$ and $Z$ given $Y$ is:

$$p(x,z \mid y) = \frac{p(x,y,z)}{p(y)} = \frac{p(z \mid y) p(y \mid x) p(x)}{p(y)}$$

Now we can follow the steps in the book. Since $p(y \mid x) p(x) = p(x,y)$ we have:

$$p(x,z \mid y) = \frac{p(x,y) p(z \mid y)}{p(y)}$$

Note that $\frac{p(x,y)}{p(y)} = p(x \mid y)$ so:

$$p(x,z \mid y) = p(x \mid y) p(z \mid y)$$

Therefore, $X$ and $Z$ are conditionally independent given $Y$.

$\endgroup$
1
$\begingroup$

It says: "first consider a chain structure: x -> y -> z (here y is observed), then:

This is a crucial statement, because as @user20160 pointed out, the relationship the book is proving is not true in general.

The important assumption that this proof departs from (and which isn't always true) is that $p(z|x,y)=p(z|y)$. In words, this means that $z$ is independent of $x$ when conditioned on $y$ (i.e. there is nothing we can learn about $z$ by observing $x$, if we already know $y$). In terms of the chain structure, this is true because there is no separate link from $x$ to $z$; the only path is through $y$. If there were a separate path from $x$ to $z$, the proof would break down.

What they end up proving is a slightly different conditional independence relationship, namely that $p(x,z|y)=p(x|y)p(z|y)$. It may be very obvious to you why this should follow from the initial assumption, and that might make it seem like they're using circular logic, but they aren't actually the same thing mathematically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.