In "Causal Inference in Statistics" (Pearl et al.), Conditional Independence is defined as (page 10): enter image description here

Later on, in the context of colliders, Conditional Dependence is defined as (page 41): enter image description here

and an example is provided for that:

Consider a simultaneous (independent) toss of two fair coins and a bell that rings whenever at least one of the coins lands on heads. Let the outcomes of the two coins be denoted X and Y, respectively, and let Z stand for the state of the bell, with Z = 1 representing ringing, and Z = 0 representing silence.

+ an extra condition is later added (page 43)

Suppose we do not hear the bell directly, but instead rely on a witness who is somewhat unreliable; whenever the bell does not ring, there is 50% chance that our witness will falsely report that it did. Letting W stand for the witness’s report

The authors claim and prove that conditioning on W makes X and Y dependent: enter image description here

(full probability table provided at the bottom)

I understand that adding Y="Heads" to the 2nd expression impacts the probability thus showing that the two variables X and Y are now dependent given W.

But, we also then get that P(X = "Heads"| Y="Heads", W = 1) is equal to P(X = "Heads"| Y="Heads") (=0.5) which is equation 1.2 of conditional independence... so how can it be? that the variables are both conditionally independent and dependent?

The probability table:

enter image description here


1 Answer 1


The difference is in which variables are independent.

  • X is independent of W, conditional on Y.
  • X is dependent on Y, conditional on W.

These statements express different relationships. There is no contradiction between them.

I will explain a little further to address your comment on Dilip Sarwate's answer. You ask, "would you say {X="Heads"|Y="Heads", W=1} is dependent or independent?" But that expression isn't something that can be dependent or independent. (I'm not even sure how to interpret it, because when we talk about events outside the context of probability statements, we can't condition on events, we can only conjoin them. For example you could write {X="Heads" & Y="Heads" & W=1}, and that would make sense.)

There are three entities involved in conditional (in)dependences: the two variables that are (in)dependent - call them A and B - and the set of variables conditioned on - call it C.

To know whether A and B are dependent or independent conditional on C, you test the equality between two probability statements:

P(A|B, C) =?= P(A|C)

Both probability statements are relevant; you are testing whether they are equal. If you change the statement on the right to P(A|B) then you are testing a different equality, a different conditional independence, and you can get a different answer.

In your example, you have

$P(X|Y, W) \neq P(X|W)$, showing that X is dependent on Y given W;

But also

$P(X|Y, W) = P(X|Y)$, showing that X is independent of W given Y.

  • $\begingroup$ Your answer makes this so clear, many thanks Lizzie! $\endgroup$
    – oshi2016
    Commented Jul 27, 2018 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.