# An example of r.v.s such that their distribution has more (conditional) independencies than their directed graphical model

I was trying to form an example where I had 3 r.v.s such that the distribution describing them had more conditional independencies or independencies than the directed graphical model corresponding to it.

I was told that if we let $X_1, X_2$ be two independent fair coin flips (WLOG 1 indicating heads, 0 indicating tails) and then define $X_3 = X_1 \oplus X_2$ i.e. be the XOR of the result of the coins. Consider its directed graphical model: I believe that the directed graphical model has less (conditional and marginal) independencies than its distribution. Does someone understand why is this case? Can someone provide me with a rigorous explanation of why this example works? If this is not the case, can someone provide me with a different example? Its ok if it has more than 3 r.vs. If it only involves fair coins that would be even better!

In general, how do you form this kind of example? What is the intuition behind it? Its not 100% intuitive for me why this should work or why its obvious or how do you come up with it.

## 1 Answer

Your example works. In the distribution, $X_1$ is marginally independent of $X_3$. That is, knowing the value of $X_3$ (but not $X_2$) does not change the distribution of $X_1$, and vice versa. But the diagram does not show this. According to the diagram, knowing $X_3$ could change the distribution of $X_1$.

To see this mathematically: $$p(X_1=1) = 0.5 \qquad p(X_3=1) = 0.5 \\ p(X_1=1, X_3=1) = p(X_1=1,X_2=0,X_3=1) = 0.5^2 \\ p(X_1=1 | X_3=1) = \frac{p(X_1=1, X_3=1)}{p(X_3=1)} = 0.5 = p(X_1=1)$$ and similarly for all values of $X_1$ and $X_3$. The key step is realizing that $X_2$ can only have one value once $X_1,X_3$ are specified, so that $p(X_1,X_3)$ is always $0.5^2$.

• what I was looking for was more of a why this example worked. Do you mind provide some of the details? Knowing just if its true or false was not really what I was looking for. – Pinocchio Oct 1 '14 at 15:16
• Sure, I've added the details. – Tom Minka Oct 1 '14 at 17:29