Understanding d-separation theory in causal Bayesian networks

Question

I am trying to understand the d-Separation logic in Causal Bayesian Networks. I know how the algorithm works, but I don't exactly understand why the "flow of information" works as stated in the algorithm.

For example in the graph above, lets think that we are only given X and no other variable has been observed. Then according to the rules of d-separation, the information flow from X to D:

1. X influences A, which is $P(A)\neq P(A|X)$. This is OK, since A causes X and if we know about the effect X, this affects our belief about the cause A. Information flows.

2. X influences B,which is $P(B)\neq P(B|X)$. This is OK, since A has been changed by our knowledge about X, the change at A can influence our beliefs about its cause, B, as well.

3. X influences C,which is $P(C)\neq P(C|X)$. This is OK because we know that B is biased by our knowledge about its indirect effect, X, and since B is biased by X, this will influence B's all direct and indirect effects. C is a direct effect of B and it is influenced by our knowledge about X.

Well, up to this point, everything is OK for me since the flow of the information occurs according to intuitive cause-effect relationships. But I don't get the special behavior of so called "V-structures" or "Colliders" in this scheme. According to the d-Separation theory, B and D are the common causes of C in the graph above and it says that if we did not observe C or any of its descendants, the flow information from X is blocked at C. Well, OK, but my question is why?

From the three steps above, started from X, we saw that C is influenced by our knowledge about X and the information flow occurred according to the cause-effect relationship. The d-Separation theory says that we cannot go from C to D since C is not observed. But I think that since we know that C is biased and D is a cause of C, D should be affected as well while the theory says the opposite. I am clearly missing something in my thinking pattern but can't see what it is.

So I need an explanation of why the flow of information blocked at C, if C is not observed.

Answer 1

Is it not intuitive that you cannot reason from cause to unobserved effect to another cause? If the rain (B) and the sprinkler (D) are causes of the wet ground (C), then can you argue that seeing rain implies that the ground is probably wet, and continue to reason that the sprinkler must be on since the ground is wet?! Of course not. You argued that the ground was wet because of the rain — you can't look for additional causes!

If you observe the wet ground, of course the situation changes. Now you may be able to reason from one cause to the other as Frank explains.

Answer 2

Let's forget about X for a moment and consider just the collider of B, C and D. The reason that the v-structure can block the path between B and D is that, in general, if you have two independent random variables (B and D) that affect the same outcome (C), then knowing the outcome can allow you to draw conclusions about the relationship between the random variables, thus allowing for information flow.

Using an example from Pearl's book on causality, let C be the observation that the lawn is wet, D the event that it rained, and B the event that the sprinkler was on. Then if you don't know whether the lawn is wet or not, C and D are clearly independent (no information flows). But if you do know that the lawn is wet, then if it rained, it's less likely that the sprinkler was on ($P(B|D) \neq P(B)$) and if the sprinkler was on, it's less likely that the wet grass was caused by the rain ($P(D|B) \neq P(D)$). Hence, knowing that the lawn is wet unblocks the path and makes B and D dependent.

To understand this better, it might be useful to have a look at Berkson's Paradox, which describes the same situation.

Answer 3

Well, up to this point, everything is OK for me since the flow of the information occurs according to intuitive cause-effect relationships. But I don't get the special behavior of so called "V-structures" or "Colliders" in this scheme.

Then the hard nut to crack here is the v-structure. I'd like to illustrate the difference between the probability of a variable S conditioned only on the observation of the effect and the influence of the observation of another variable D which is independent of S in the same situation using a fictitious example.

Let's say that someone is taking a course, say linear algebra. If he can pass it mainly dependens on the difficulty of the exam. Let's denote the event of passing the course by P, passing as 1 and 0 otherwise; and the difficulty of the exam as D, difficult as 1 and easy as 0. And something nonsense may also exert an influence on his performance or the result, let's say the singularity happens and he would be brainwashed by a machine and then decides not to take the exam. We denote that event by S, and its probability is 0.0001. That seems impossible but by definition its chance should not be zero.

Hence we have a graph of the v-structure form now:

 D   S
  | |
 \| |/ 
   P  

If we know that if sigularity comes the student doesn't take the exam: $P(\neg P|S) = 0.999999$ and $P(P|S)=0.000001$, no matter how easy the exam would be. And the prior probabilities are as follows:

| d0   | d1      |      
|:-----|--------:|   
| 0.5  | 0.5     |  

| s0     | s1      |      
|:-------|--------:|   
| 0.9999 | 0.0001  |

| S     | D    | P(p0|S,D) | P(p1|S,D) |  
|:------|-----:|----------:|----------:|
|s0     | d0   |   0.20    |   0.80    |
|s0     | d1   |   0.90    |   0.10    |
|s1     | d0   |   0.999999|   0.000001|
|s1     | d1   |   0.999999|   0.000001| 

To check if S and D are independent or not given P we should work out two distributions(see the first two equations in wikipedia: Conditional Independence): $P(S|P)$ and $P(S|P, D)$. If they are equal we can say that the conditional independence hold, otherwise it doesn't.

1) If we don't know the result we can calculate the probability of the singularity happenning given the course is easy.

\begin{align} P(S|\neg D) & = P(S, P|\neg D)+P(S, \neg P| \neg D) \\ & = \frac{P(S=1, P=1, D=0)}{P(D=0)} + \frac{P(S=1, P=0, D=0)}{P(D=0)} \\ & = \frac{P(S=1)P(D=0|S=1)P(P=1|D=0,S=1)}{P(D=0)} + \frac{P(S=1)P(D=0|S=1)P(P=0|D=0,S=1)}{P(D=0)} \\ & = \frac{P(S=1)P(D=0|S=1)}{P(D=0)} \\ & = \frac{P(S=1)P(D=0)}{P(D=0)} \\ & = P(S=1) \\ & = 0.0001 \end{align}

As you can see above that doesn't matter if the exam is passed or not. What comes as it should come. It can be seen as a marginal probability over P.

And we can also work out the probability the the singularity happens given that the student doesn't pass the exam:

\begin{align} P(S, |\neg P) &= \frac{P(S,\neg P)}{P(\neg P)} \\ &= \frac{P(S,\neg p, D) + P(S,\neg P, \neg D)}{P(\neg P)}\\ &= \frac{P(\neg P|S, D) P(S) P(D)+P(\neg P|S, \neg D)P(S)P(\neg D)}{\sum_{S,D}P(\neg P |S,D)P(S)P(D) }\\ &= 0.0001818 \end{align}

Knowing that the guy doesn't pass the exam we can guess that he may be brainwashed by a machine is 0.0001818 which is a little bigger than when we don't know it.

2) But what if we know that the guy failed the exam and the exam is easy? \begin{align} P(S, |\neg P, \neg D) &= \frac{P(S=1, P=0, D=0)}{P(P=0, D=0)} \\ & = \frac{P(P=0|S=1, D=0)P(S=1)P(D=0)}{P(P=0|S=1, D=0)P(S=1)P(D=0)+P(P=0|S=0, D=0)P(S=0)P(D=0)} \\ & = \frac{0.999999 \times 0.0001 \times 0.5}{0.2 \times 0.9999 \times 0.5+0.999999 \times 0.0001 \times 0.5} \\ & = 0.0004998 \end{align}

Lo and behold, the change is much bigger than we just know he doesn't plass the exam. Then we see that $P(S|P) \neq P(S|P, D)$ we can infer that $S \perp D | P \notin I(P(P, S, D))$ which means D can influence S via P.

May this detailed derivation be of hlep.