1
$\begingroup$

I was trying to form an example where I had 3 r.v.s such that the distribution describing them had more conditional independencies or independencies than the directed graphical model corresponding to it.

I was told that if we let $X_1, X_2$ be two independent fair coin flips (WLOG 1 indicating heads, 0 indicating tails) and then define $X_3 = X_1 \oplus X_2$ i.e. be the XOR of the result of the coins. Consider its directed graphical model:

enter image description here

I believe that the directed graphical model has less (conditional and marginal) independencies than its distribution. Does someone understand why is this case? Can someone provide me with a rigorous explanation of why this example works? If this is not the case, can someone provide me with a different example? Its ok if it has more than 3 r.vs. If it only involves fair coins that would be even better!

In general, how do you form this kind of example? What is the intuition behind it? Its not 100% intuitive for me why this should work or why its obvious or how do you come up with it.

$\endgroup$
1
$\begingroup$

Your example works. In the distribution, $X_1$ is marginally independent of $X_3$. That is, knowing the value of $X_3$ (but not $X_2$) does not change the distribution of $X_1$, and vice versa. But the diagram does not show this. According to the diagram, knowing $X_3$ could change the distribution of $X_1$.

To see this mathematically: $$ p(X_1=1) = 0.5 \qquad p(X_3=1) = 0.5 \\ p(X_1=1, X_3=1) = p(X_1=1,X_2=0,X_3=1) = 0.5^2 \\ p(X_1=1 | X_3=1) = \frac{p(X_1=1, X_3=1)}{p(X_3=1)} = 0.5 = p(X_1=1) $$ and similarly for all values of $X_1$ and $X_3$. The key step is realizing that $X_2$ can only have one value once $X_1,X_3$ are specified, so that $p(X_1,X_3)$ is always $0.5^2$.

$\endgroup$
  • $\begingroup$ what I was looking for was more of a why this example worked. Do you mind provide some of the details? Knowing just if its true or false was not really what I was looking for. $\endgroup$ – Pinocchio Oct 1 '14 at 15:16
  • $\begingroup$ Sure, I've added the details. $\endgroup$ – Tom Minka Oct 1 '14 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.