In Causality - Models, Reasoning, And Inference by Pearl, definition 2.3.3 reads as follows -

One latent structure $L$ = $\langle D,O \rangle$ is preferred to another $L^{'}$ = $\langle D^{'},O \rangle$ (written $L \preceq L^{'}$) if and only if $D^{'}$ can mimic $D$ over $O$ - that is, if and only if for every $\Theta_{D}$ there exists a $\Theta^{'}_{D^{'}}$ such that $P_{[O]}(\langle D^{'},\Theta^{'}_{D^{'}}\rangle)=P_{[O]}(\langle D,\Theta_{D} \rangle)$. Two latent structures are equivalent, written $L^{'} \equiv L$ if and only if $L \preceq L^{'}$ and $L \succeq L^{'}$.

In what sense is $L$ preferred over $L^{'}$? Is it because the probability distributions satisfied by $L$ are a subset of those satisfied by $L^{'}$?

In particular, please refer to the below example Pearl cites later -

enter image description here

For the independencies $(a \perp b)$, $(d \perp \{a,b\} \ | \ c)$, figure (a) captures all of them. Figure (c) does not necessitate $(a \perp b)$.

Does it mean that figure (c) can be made to fit not only all probabilities where $(a \perp b)$ holds, but also others where it does not hold? Is figure (a) preferred over (c) in this sense, given the independencies $(a \perp b)$, $(d \perp \{a,b\} \ | \ c)$?

Note: I am familiar with minimal I-maps, perfect maps etc., from Probabilistic Graphical Models by Koller. If necessary, one can answer in those terms. I have a hunch that figure (a) is a perfect map while (c) is a minimal I-map for the independencies cited; that might be the reason for preferring (a) over (c). Am I correct?


1 Answer 1


Definition 2.3.3 is in essence a statement about "excess edges" or "excess dependencies"

Let us for a second assume there are no hidden variables. Then, a structure $L'$ that can, with the right parametrization $\Theta'_{D'}$, mimic all probability distributions of an alternative structure $L$ (such that $P_{[O]}(\langle D', \Theta'_{D'}\rangle) = P_{[O]}(\langle D, \Theta_{D}\rangle)$, must contain all edges present in $L$ and possibly more (not necessarily the same direction). In this case, the structure with fewer or equal edges $L$ is preferred. So if we denote the number of edges in $L$ as $|D|$, we could read Definition 2.3.3 as saying that we prefer $L$ over $L'$ if $|D| \leq |D'|$. We say $L \equiv L'$ if $|D| = |D'|$.

This isn't quite true when there are hidden variables (more on that below), but I think it gives good intuition as to why we would want to have this definition. In terms of Occams razor we prefer the model with the fewest assumption (fewest edges) among models that fit the data equally well.

Figure 2.1

Figure 2.1 illustrates the concept by showing a minimal model a), another minimal model b) including a potential hidden variable, and a model with an excess edge c).

The model in a) appears to be a graph that fits all distributions $P_{[O]}$ without any unnecessary dependencies between observed variables.

The model in b) is basically the same model as a), except that a hidden variable is assumed to explain the dependence $a \to c$. Practically speaking, such a model is not very useful because a hidden variable may explain all distributions and can never be ruled out (see first paragraph in 2.3). So although the number-of-edges reasoning does not apply here directly because we have more variables to begin with, we could re-phrase the same idea using the number of dependence statements between observed variables. In this sense, b) is equivalent to a).

The model in c) contains an "excess edge" in $a \to b$. The distribution $P_{[O]}$ can be explained without it as seen in a).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.