2
$\begingroup$

Suppose we have a simple Bayesian Network as follows: $X_1$ --> $X_3$ <-- $X_2$. Using the chain rule of Bayesian Networks, we can say the following:

$$ f(x_1,x_2,x_3) = f(x_1) f(x_2) f(x_3 | x_1, x_2) \ \ \ \ \ \ \ \ \ \ \ \ [1]$$

We know that

$$ f(x_3 | x_1, x_2) = \frac{f(x_3,x_1,x_2)}{f(x_1,x_2)} \ \ \ \ \ \ \ \ \ \ \ [2]$$

Thus, for Equation 1 above to be valid, $f(x_1,x_2)$ must be equal to $f(x_1) * f(x_2)$, implying that $X_1$ must be independent to $X_2$.

I want to confirm that the following logic for $X_1$ being independent to $X_2$ is correct:

In the Directed Acyclic Graph (DAG) above, $X_1$ is indeed independent to $X_2$ because of d-separation. $X_1$ and $X_2$ are d-separated as long as $X_3$ is not observed, which is the scenario when computing $f(x_1,x_2)$.

This seems somewhat subtle to me, I haven't seen this explicitly stated in any notes that I found on the Internet ...

Improve this question
$\endgroup$
2
  • $\begingroup$ You are right. The DAG implies that $X_1$ and $X_2$ only become dependent when conditioning on $X_3$. $\endgroup$ – HStamper May 9 '16 at 16:57
  • $\begingroup$ Eric has already answered the question. You may want to have a look at the concept of explaining away which is immediately relevant to: X1 and X2 are only related through X3. $\endgroup$ – Zhubarb May 12 '16 at 16:03
1
$\begingroup$

Your logic sounds correct, however I would like to add a few things.

  • The chain rule results in $f(x_1|x_2,x_3)f(x_2|x_3)f(x3)$, basically its the entire parameter space.

The DAG $X_1 \rightarrow X_3 \leftarrow X_2$ will fall under 1 of these 2 cases:

  • if $X_3$ is not in the conditioning set $Z$, the path (that's all paths in this case) from $X_1$ to $X_2$ will be inactive, therefore we can say $X_1\!\perp\!\!\!\perp X_2 | X_3 \implies P(X_1,X_2,X_3) =P(X_1|X_3)*P(X_2|X3)$
  • if $X_3$ or any of its' descendants are in the conditioning set $Z$, we have an active path, so we cannot guarantee independence as in the first case, note however that this does not imply the opposite.
Improve this answer
$\endgroup$
3
  • $\begingroup$ I've edited the question to say "chain rule of Bayesian networks," which is different than the chain rule of probability distributions in general. I do not believe that $X_1$ and $X_2$ will always be d-separated. The rules of d-separation state that $X_1$ and $X_2$ are only d-separated if $X_3$ (or any of its descendants, which is not relevant to the example above) are not observed. If $X_3$ is observed, the trail becomes active and $X_1$ and $X_2$ are dependent. $\endgroup$ – Kiran K. May 9 '16 at 17:47
  • $\begingroup$ Actually, I think I made this more complicated than it needs to be. In this DAG, $X_1$ is always independent of $X_2$. For any variable $X_i$ in a DAG, $X_i$ is independent of its NonDescendant's given its parents. In this case, $X_1$ is independent of $X_2$, given $X_1$'s parents, which is null (and similarly for $X_2$). perhaps thats what you were saying with $d1$ and $d2$ being d-separated regardless of $d3$? $\endgroup$ – Kiran K. May 12 '16 at 15:00
  • $\begingroup$ you're right, I've updated my answer accordingly, $\endgroup$ – Edqu3 May 12 '16 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.