Explanation of I-map in a Markov/Bayesian network

Question

I am finding the concept of an I-map (Independency-map) in the context of Markov networks and Bayesian networks difficult to understand. From Probabilistic Graphical Models, Koller and Friedman, 2009:

We first define the set of independencies associated with a distribution $P$.

Let $P$ be a distribution over $X$. We define $I(P)$ to be the set of independence assertions of the form $(X \bot Y|Z)$ that hold in $P$.

We can now rewrite the statement that "$P$ satisfies the local independencies associated with $G$" simply as $I(G) \subseteq I(P)$. In this case, we say that $G$ is an I-map for $P$.

Where I'm getting confused:

• Does this represent every possible independence in a graph given every possible subset of the set of variables $Z$? Or are you able to define $I(P)$ because the graph structure is specified?

• The whole last sentence is confusing to me. Maybe it's because I can't think abstractly enough, but I don't understand how the I-map of G is a subset of the I-map of P.

Any help is appreciated!

Answer 1

From my understanding, if a DAG G is said to be the I-Map of probability distribution P, then every independence we can observe from G is encoded in P. Let's consider a simple example:
Suppose distribution $P_1$ has independence $\{(I\perp D)_p\}$, and distribution $P_2$ has no independence, or $\emptyset$.
Now we define two DAGs: $G$ and $G'$

$G$ is I-Map of $P_1$ because $I$ and $D$ are independent in both $G$ and $P_1$. $G$ is not I-Map of $P_2$ because $P_2$ fails to satisfy the independence between I and D.
(Surprisingly?) $G'$ is I-Map of both $P_1$ and $P_2$ because the independence in $G'$ is $\emptyset$. Since $\emptyset$ is a subset of every set, both $P_1$ and $P_2$ satisfy the independence in $G'$.
Therefore, I-Map, in plain words, means that the independence shown in a DAG is a subset of the independence shown in a distribution.
Hope this helps to clear it up a bit for you.

Answer 2

Does this represent every possible independence in a graph given every possible subset of the set of variables $Z$? Or are you able to define $I(P)$ because the graph structure is specified?

No. The set of all possible conditional independencies expressed by a DAG $I(G)$ is different from the set of all possible conditional independencies we can find in a certain joint distribution $I(P)$.

The whole last sentence is confusing to me. Maybe it's because I can't think abstractly enough, but I don't understand how the I-map of G is a subset of the I-map of P.

Normally $I(G)\subset I(P)$ which means the set of independencies we can see from the connectivity in the graph is only a part of the independencies the joint distribution has, which indicates the soundness rather than the completeness of the d-separation. All independencies we can get from the graph easily are right and can be verified in the joint distribution but some dependencies/edges are redundant. If we don't use the graph to visually express the independencies it would be much harder for us to tackle the related problems by directly checking the joint distribution.

The use of Bayes Network is expressing conditional independence and the more conditional independencies we can express using the graph for the joint distribution we are dealing with the better. Since almost everything in the universe is to some extent dependent on each other in some way, and we can just simplify the issue by assuming some independencies, otherwise, we cannot tackle any problem.

An easy example: for every fully connected graph $I(G)=\emptyset$ and every $\emptyset$ is a subset of the set of independencies in any joint distribution and thus this always holds: $I(G)\subset I(P)$. But the graph is totally unrepresentative and useless because it tells us nothing about the independence structure in the distribution.

If $I(G)=I(P)$ the graph is a perfect graph: P-Map, which means all independencies can be perfectly expressed by the graph( and all the independencies in the graph are right for the joint distribution).