# Chain rule for Bayesian Networks

Suppose we have a simple Bayesian Network as follows: $X_1$ --> $X_3$ <-- $X_2$. Using the chain rule of Bayesian Networks, we can say the following:

$$f(x_1,x_2,x_3) = f(x_1) f(x_2) f(x_3 | x_1, x_2) \ \ \ \ \ \ \ \ \ \ \ \ [1]$$

We know that

$$f(x_3 | x_1, x_2) = \frac{f(x_3,x_1,x_2)}{f(x_1,x_2)} \ \ \ \ \ \ \ \ \ \ \ [2]$$

Thus, for Equation 1 above to be valid, $f(x_1,x_2)$ must be equal to $f(x_1) * f(x_2)$, implying that $X_1$ must be independent to $X_2$.

I want to confirm that the following logic for $X_1$ being independent to $X_2$ is correct:

In the Directed Acyclic Graph (DAG) above, $X_1$ is indeed independent to $X_2$ because of d-separation. $X_1$ and $X_2$ are d-separated as long as $X_3$ is not observed, which is the scenario when computing $f(x_1,x_2)$.

This seems somewhat subtle to me, I haven't seen this explicitly stated in any notes that I found on the Internet ...

• You are right. The DAG implies that $X_1$ and $X_2$ only become dependent when conditioning on $X_3$. – HStamper May 9 '16 at 16:57
• Eric has already answered the question. You may want to have a look at the concept of explaining away which is immediately relevant to: X1 and X2 are only related through X3. – Zhubarb May 12 '16 at 16:03

• The chain rule results in $f(x_1|x_2,x_3)f(x_2|x_3)f(x3)$, basically its the entire parameter space.
The DAG $X_1 \rightarrow X_3 \leftarrow X_2$ will fall under 1 of these 2 cases:
• if $X_3$ is not in the conditioning set $Z$, the path (that's all paths in this case) from $X_1$ to $X_2$ will be inactive, therefore we can say $X_1\!\perp\!\!\!\perp X_2 | X_3 \implies P(X_1,X_2,X_3) =P(X_1|X_3)*P(X_2|X3)$
• if $X_3$ or any of its' descendants are in the conditioning set $Z$, we have an active path, so we cannot guarantee independence as in the first case, note however that this does not imply the opposite.
• I've edited the question to say "chain rule of Bayesian networks," which is different than the chain rule of probability distributions in general. I do not believe that $X_1$ and $X_2$ will always be d-separated. The rules of d-separation state that $X_1$ and $X_2$ are only d-separated if $X_3$ (or any of its descendants, which is not relevant to the example above) are not observed. If $X_3$ is observed, the trail becomes active and $X_1$ and $X_2$ are dependent. – Kiran K. May 9 '16 at 17:47
• Actually, I think I made this more complicated than it needs to be. In this DAG, $X_1$ is always independent of $X_2$. For any variable $X_i$ in a DAG, $X_i$ is independent of its NonDescendant's given its parents. In this case, $X_1$ is independent of $X_2$, given $X_1$'s parents, which is null (and similarly for $X_2$). perhaps thats what you were saying with $d1$ and $d2$ being d-separated regardless of $d3$? – Kiran K. May 12 '16 at 15:00