Simpson's paradox in Judea Pearl's book?

Question

I'm looking at the following question in Judea Pearl's primer on causality In an attempt to estimate the effectiveness of a new drug, a randomized experiment is conducted. In all, 50% of the patients are assigned to receive the new drug and 50% to receive a placebo.

A day before the actual experiment, a nurse hands out lollipops to some patients who show signs of depression, mostly among those who have been assigned to treatment the next day (i.e., the nurse’s round happened to take her through the treatment-bound ward). Strangely, the experimental data revealed a Simpson’s reversal: Although the drug proved beneficial to the population as a whole, drug takers were less likely to recover than nontakers, among both lollipop receivers and lollipop nonreceivers. Assuming that lollipop sucking in itself has no effect whatsoever on recovery, answer the following questions:

1. Is the drug beneficial to the population as a whole or harmful?

2. Does your answer contradict our gender example, where sex-specific data was deemed more appropriate?

3. Draw a graph (informally) that more or less captures the story.

4. How would you explain the emergence of Simpson’s reversal in this story?

5. Would your answer change if the lollipops were handed out (by the same criterion) a day after the study?

[Hint: Use the fact that receiving a lollipop indicates a greater likelihood of being assigned to drug treatment, as well as depression, which is a symptom of risk factors that lower the likelihood of recovery.]

A solution to this was posted before here Distribution to match an example with collider bias?

With the following code

    set.seed(10)
    n     <- 1e6
    x     <- rbinom(n, 1, 0.5)
    dep   <- rbinom(n, 1, 0.5)
    lolli <- rbinom(n, 1, 0.5*x + 0.5*dep)
    y     <- rbinom(n, 1, 0.1*x + 0.8*dep)

    mean(y[x==1]) - mean(y[x==0]) # 0.1
    mean(y[x==1 & lolli==1]) - mean(y[x==0 & lolli==1]) # -0.17
    mean(y[x==1 & lolli==0]) - mean(y[x==0 & lolli==0]) # -0.17

I've run the code and seen that it gives the desired results. However one thing confuses me is that in the hint it states " drug treatment, as well as depression, which is a symptom of risk factors that lower the likelihood of recovery".

But our target variable $y \sim \mathrm{Ber}(0.1x + 0.8 \times \text{dep})$ which clearly shows that if $x = 1$ and $\text{dep} = 1$ then we have a 90% chance of recovery? In fact here being depressed increases your chance of recovery but shouldn't it be the opposite? Is this a mistake or have I misunderstood something?

Answer 1

As described, you are correct that the observed phenomenon is impossible (assuming a simply DAG with only treatment ($T$), depression ($D$), lollipop ($L$), and recovery ($Y$).

Conditioning on being given a lollipop opens a backdoor path from treatment to the outcome through depression; this is the cause of the collider bias we witness in the Simpson's reversal. Taking the description as given, we expect \begin{aligned} \beta_{TY} &> 0\\ \beta_{TL} &> 0 \\ \beta_{DL} &> 0 \\ \beta_{DY} &<0 \end{aligned} where $\beta_{TL}$ is the effect of $Y$ on $Y$, etc., in a linear structural equation model for the system as was implemented in your R code. The true effect of $T$ on $Y$ is $\beta_{TY} > 0$, but the implication of conditioning on the collider is that $\beta_{TY|L=l} < 0$. With path analysis tools, we can see that as described, this can never be true.

Carlos Cinelli provides a formula for computing the relationship between two antecedents of a conditioned-upon collider here. Importantly, when the antecedents are unconditionally independent (as they are here), the sign of their relationship is opposite of the product of the signs of their relationships with the collider (i.e., $\beta_{TL}$ and $\beta_{DL}$). Because $\beta_{TL}$ and $\beta_{DL}$ are both positive, the conditonal association between $T$ and $D$ is negative. Because $\beta_{DY}$ is negative, the backdoor path from $T$ to $Y$ through $D$ when conditonng on $L$ is positive, the same direction as the direct effect of $T$ on $Y$. This implies that the total association between $T$ and $Y$ will be positive when conditioning on $L$ (summing the two pathways $\beta_{TD|L}\beta_{DY} + \beta_{TL}$), which is the opposite of what we are told in the story (which is that the conditional effect of $T$ on $Y$ is negative, $\beta_{TY|L=l} < 0$). So the data-generating model as described does not yield the phenomenon described (i.e., the reversal of signs).

To make it so, the only thing that needs to change is that the control group is more likely to be visited by the nurse (i.e., $\beta_{TL} < 0$). Then, it is possible to witness the phenomenon as described. In R, the simple fix is to replace

lolli <- rbinom(n, 1, 0.5*x + 0.5*dep)

with

lolli <- rbinom(n, 1, 0.5*(1-x) + 0.5*dep)

and the phenomenon is observed with the same values (i.e., an unconditional effect of .1 and a conditional effect of -.17).