6
$\begingroup$

I was reading Koller's and Friedman's Probabilistic Graphical Models book and became confused about some of its notation because of a set of notes that either contradict it or express it differently. Whichever it is, I would like to clarify.

The notation is:

$(\textbf{X} \perp \textbf{Y} , \textbf{W}\mid \textbf{Z})$

The reason I got confused on what the notation means is because of the following slides online:

http://www.stats.ox.ac.uk/~steffen/seminars/waldmarkov.pdf

Particularly what confuses me is the way these notes express the decomposition rule on page 8 and how koller's book expresses this same decomposition rule a little differently. If I understand the two notations, I should understand the decomposition rule under both notations.

Koller's book expresses the decomposition rule as follow:

$$(\textbf{X} \perp \textbf{Y} , \textbf{W}\mid \textbf{Z}) \implies (\textbf{X} \perp \textbf{Y}\mid \textbf{Z}, \textbf{W})$$

Which I would express intuitively as; the set of random variables $\textbf{X}$ are independent of the set of random variables $\textbf{Y}$ and $\textbf{W}$ (i.e. interpreting the comma in between $\textbf{Y}$ and $\textbf{X}$ as an "and") given the set of random variables $\textbf{Z}$.

However, the slides seem to interpret this comma separation as an Union which confuses me (because I thought we were doing "ands"):

$$(A \perp_{\sigma} (B \cup D) \mid C) \implies (A \perp_{\sigma} B \mid C) \text{ and } (A \perp_{\sigma} D \mid C) $$

It confuses me because they seem to use union for the commas were I used "ands". Furthermore, I can't see why the two statements are equivalent decomposition rules (which might explain why I've had a tough time proving it because I don't think I understand the statement!). Does someone know the difference in the notation and how they equivalently express the decomposition rule?


As a reference this is more background on the notation in Koller's Book:

Let capital non-bold stand for random variables say $X$ is a r.v. Let little non bold stand for the assignment to a random variable say $(X = x)$. Also, let me define captial bold letters as sets of random variables. For example $\textbf{X}, \textbf{Y}, \textbf{Z}$ are three sets of random variables. Let small bold letters denote assigments to these sets $\textbf{x}, \textbf{y}, \textbf{z}$ i.e. it denotes assigments of values to the variables in these sets. Let $Val(\textbf{X})$ be the values that the set of random variables can take.

On page 21 of Koller's book it explicitly says that Intersections are comma separated, so its fair to assume comma's means "intersection". This can be appreciated on page 21 where it says:

Rather than write $P((X=x) \cap (Y=y))$, we write $P(X=x, Y = y)$ or just $P(x,y)$.

I know this is only on the context of explicit probability distributions, however, if we note how it starts to prove the decomposition property, we can infer that koller probably mean a comma as a intersection in the confusing notation $(\textbf{X} \perp \textbf{Y} , \textbf{W}\mid \textbf{Z})$ too. She starts like this:

For example, to prove Decomposition assume that $(X \perp Y , W \mid Z)$ holds. Then, from the definition of conditional independence, we have that $P(X,Y,W|Z) = P(X|Z)P(Y,W|Z)$. Now using... proof continues...

The equation:

$$P(X,Y,W|Z) = P(X|Z)P(Y,W|Z)$$

Makes me suspect that the slides are either wrong or sloppy in notation. Or both are sloppy. I am not sure.

$\endgroup$
  • $\begingroup$ I was looking for this exact formula today! Good to know I am no the only one confused about the notation. Too bad the question still does not have a proper answer... $\endgroup$ – Jan Kukacka Feb 27 '18 at 20:18
3
$\begingroup$

The interpretation of the comma depends on context. In the case

$$A \perp B, C$$

which is a statement about (un-)conditional dependencies, it means "$A$ is independent of the combined set of events $B \cup C$".

In the case,

$$P(A = a, B = b)$$

which is about specific events, it means "the probability of both $a$ and $b$ occurring together", ie $P(A=a) \cap P(B=b)$.

I think this corresponds with how most people would intuitively interpret the comma in both situations. I certainly didn't find anything ambiguous or sloppy in the examples given.

If there's any ambiguity, I'd say it's in the order of precedence for the various symbols and punctuation marks. At first glance I thought $\textbf{X} \perp \textbf{Y} , \textbf{W}\mid \textbf{Z}$ meant $(\textbf{X} \perp \textbf{Y})$, followed by $(\textbf{W}\mid \textbf{Z})$....

$\endgroup$
2
$\begingroup$

The puzzling equation appears to be

$$(A \perp_{\sigma} (B \cup D) \mid C) \implies (A \perp_{\sigma} B \mid C) \text{ and } (A \perp_{\sigma} D \mid C)$$

Try to write the left-hand side as an intersection (an "and")

$$(A \perp_{\sigma} (B \cap D) \mid C) \implies (A \perp_{\sigma} B \mid C) \text{ and } (A \perp_{\sigma} D \mid C) \qquad ??$$

Is it now true? In the left-hand side, only the parts of $B$ and $D$ that coincide appear, and the relation of this "common part" with $A$ is characterized, under the light of $C$. But in the right hand side, both $B$ and $D$ appear in their wholeness - so we can not go from the LHS to the RHS, since the left hand side remains silent about either $B$ \ $D$, or $D$ \ $B$, which appear in the right-hand side.

But in the initial statement of the presentation, we cover in the left-hand side already the whole of $B$ and the whole of $D$ by using union.

$\endgroup$
1
$\begingroup$

In Koller's book(2009), the two cases in your question are weak union and decomposition respectively.

enter image description here

The proof of the decomposition is given on the book(page 25) using the reasoning by cases, and it seems very clear that the comma represents "AND". enter image description here

For the weak union property, here is my proof. Hope it may be of any help.

We assume that $(X\perp Y, W|Z)$ holds, then we can obtain: \begin{align} P(X, Y|Z, W) &= \frac{P(X, Y, W|Z)}{P(W|Z)}\\ &= \frac{P(X|Z)P(Y,W|Z)}{P(W|Z)}\\ &= \frac{P(X|W,Z)P(Y,W|Z)}{P(W|Z)}\\ &= P(X|W, Z) P(Y|W,Z) \end{align}

Then the weak union property holds.

For the third equality, it is because of the decomposition property: if $(X|Y,W|Z)$ holds we can get $P(X,W|Z) = P(X|Z)P(W|Z)$(and hence $P(X|Z)=\frac{P(X,W|Z)}{P(W|Z)}=P(X|W,Z)$).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.