Equivalence between directed and undirected graph?

Question

I am confused over something that may have an obvious explanation I am missing.

In Koller's Probablistic Graphical models textbook, page 945, it is said that a Markov network $A-B-C$ is equivalent to a Bayesian network $A\rightarrow B\rightarrow C$, and that both have equivalent expressive power.

However, I find this confusing because in the latter, $P(B|A,C) = P(B|A)$ due to the assumption in directed graphs where a variable is conditionally independent of others given its ancestor. Isn't this independence missing in the undirected version of the model?

I'd appreciate any input, thanks!

Answer 1

The fact that they are equivalent comes down to the fact that they exhibit the same set of conditional independencies.

The mistake in your reasoning is that $𝑃(𝐵|𝐴,𝐶)\not=𝑃(𝐵|𝐴)$ in the directed graphical model - in fact, a node is only conditionally independent of all others given its Markov blanket.

In your directed graphical model example, conditioning on both $A$ and $C$ will provide more information about $B$ than just conditioning on $A$ alone.

For instance, suppose $B$ is a mixture of Gaussians where $A$ is a discrete variable labelling each mixture component, and $C = B + \epsilon$ where $\epsilon$ is a small Gaussian corruption. Then, observing $A$ amounts to knowing which mixture component $B$ will be sampled from, and additionally observing $C$ will still give more information about the value of $B$.