# Is every distribution factorizable by an MRF also factorizable via a Bayesian network? And vice versa?

This has probably been asked before, so if it has please provide a link to the original question and close this as a duplicate -- I was not able to find the original question myself.

Question: Let's say we are given a joint probability distribution $$\mathbb{P}$$ of some random variables $$X_1, \dots, X_n$$, and we know that there exists an undirected graph $$G$$ such that $$\mathbb{P}$$ is a distribution compatible with the Markov random field defined by $$G$$. Then does there exist a directed acyclic graph $$N$$ such that $$\mathbb{P}$$ is a distribution compatible with the Bayesian network defined by $$N$$?

And conversely, let's say that we know $$\mathbb{P}$$ is a distribution compatible with the Bayesian network defined by the directed acyclic graph $$N$$. Then does there exist an undirected graph $$G$$ for which $$\mathbb{P}$$ is compatible with the Markov random field defined by $$G$$?

I would also be interested in knowing whether this is true only when given some restrictions on $$\mathbb{P}$$, e.g. that it be strictly positive. I found some lecture slides talking about how we can triangulate a graph $$G$$ corresponding to a Markov random field and get something called an "I-map" that corresponds to a Bayesian network, and something similar by "moralizing" a Bayesian network to get a Markov random field. But I am interested in the statement at the level of specific, concrete probability distributions and am having difficulty translating it to that level.

• In addition to those slides, you might be interested in this tutorial introduction to conditional random fields: homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf . Section 2.1.1 motivates factor graphs as a useful formalism by pointing out that they resolve ambiguities that markov networks cannot. Jan 26 '20 at 16:07

If you have a DAG (directed acyclic graph) you can moralize it to obtain a UG (undirected graph). The distributions that are compatible with the DAG will then also be compatible with the UG. However, there might be some distributions implied by the UG that will not be compatible with the DAG.

Likewise, you can not always express the distributions implied by a UG with a DAG. See handbook of graphical models page 39-42

Edit: a directed acyclic graph (DAG) $$\mathcal{H}$$ gives rise to a family of probability densities that factorizes as $$p(x) = \prod_{v \in \mathcal{H}} p(x_v | x_{ parents(v) })$$.

An undirected graph gives rise to a family of densities that factorizes across the cliques of that graph.

The question is if we can express the same family of densities through a DAG as we can through a UG. The answer is that this is only possible when the graph is decomposable. If we for example have an UG on the form a - b - c - d - a, we cannot make a DAG where the densities implied by the DAG are the same as the densities implied by that UG.

The same is true in the other direction - it is not necessarily possible to express the densities which respect the structure implied by a DAG through a UG. However, we can moralize the DAG, and the densities that are implied by the DAG will be contained in the familiy of densities implied by the moralized DAG. However, there might be densities implied by the moralized DAG that are not in the family implied by the original DAG.

• This is so helpful, thank you so much! To clarify, what is the "handbook of graphical models"? I wanted to look at that pages you mentioned, 39-42, but am not sure which book you are referring to. Jan 28 '20 at 4:44
• You're welcome, the book I am referring to is the one written by Marloes Maathuis, Mathias Drton, Steffen Lauritzen and Martin Wainwright, a PDF version can be found here stat.ethz.ch/~maathuis/papers/Handbook.pdf
– Abm
Jan 28 '20 at 8:31
• Oh wow, thank you so much again! I've heard of most of those names, but didn't know they had all written a book together! Based on everything it seems like an important reference. Jan 29 '20 at 5:54