2
$\begingroup$

In Barber's book pp. 40-41 he says that the belief network X->Z<-Y: graph

is "graphically dependent" since:

$$p(x,y|z) \propto p(z|x,y)p(x)p(y)$$

I don't understand why graphical dependence follows from the above equation? Is it because we don't see $p(x,y|z)\propto p(x|z)p(y|z)$?

Further, in a similar example he also says that the same graph represents $p(x,y)=p(x)p(y)$, i.e. that $x$ and $y$ are independent, which I assume follow from the definition: $p(x,y,z)=p(z|x,y)p(x)p(y)$.

Is he saying that $x$ and $y$ are independent, but dependent when conditioned on $z$?

Why is $x$ and $y$ graphically dependent in the above graph?

$\endgroup$
1
  • 1
    $\begingroup$ In that picture, if an arrow connects two nodes they seem to be unconditionally dependent. Then $X$ and $Y$ are independent because no arrow connects them, explaining why $p(x,y)$ factors as $p(x)p(y)$. Using that fact and properties of conditional probability we have , $P(x,y|z)=P(x,y,z)/P(z)=P(x)p(y)p(z,x,y)/p(z)$ which is the proportionality claim. $X$ and $Y$ are unconditionally independent but conditional on $Z$ they are dependent. $\endgroup$
    – CloseToC
    Jan 19, 2015 at 22:53

1 Answer 1

5
$\begingroup$

The term "graphical dependence" seems to be specific to Barber's book. I have not seen it anywhere else. I think by graphical dependence, he means that there are arrows connecting variables together. So, the connected variables appear to be dependent. But actually they may or may not. They are just dependent (connected) graphically.

In the graph, $x\rightarrow z \leftarrow y$, you have $x \not\perp y | z$ because you cannot get $p(x|z)p(y|z)$ from $p(x,y|z)$, as you said. In words, $x$ and $y$ are not conditionally dependent given $z$. In this graph, it is true that $x\perp y$. That is, $x$ is (marginally) independent of $y$ because

$p(x,y,z) = p(x)p(y) p(z|x,y)$

$p(x, y) = \sum_z p(x,y,z) = p(x)p(y) \overbrace{\sum_z p(z|x,y)}^1 = p(x)p(y).$

This graph kind of structure where there are two arrows pointing two one variable is called a V-structure. $z$ is also called a collider node.

A good concrete example to illustrate the concept is as follows. Imagine $x$ and $y$ are two independent coins with arbitrary probabilities for heads. Let $z \in \{0, 1\}$ where $z=1$ if $x=y$ and $z=0$ if $x\neq y$. By design, $x$ and $y$ are independent. So, $x \perp y$. Now assume $z=1$. Given $z=1$, it can be seen that knowing $x$ reveals the value of $y$. If $x=1$, you can deduce that $y=1$ because $z=1$ means $x$ and $y$ are the same. Here, given $z$ you know more about $y$ by observing $x$ and vice versa. This effect is called "explaining away". Hence, $x \not\perp y | z$. In this example, we did not put any randomness in $z|x,y$. However, the principle is the same even with stochasticity.

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.