Is there an example of two causally dependent events being logically (probabilistically) independent?

Question

Two events $A,B$ are independent when $P(A \cap B ) = P(A)P(B)$ I am trying to drill into this definition and to try to reconcile it with our intuitive idea of independence in the real world. I feel that the equation can be achieved by accident, without any grounds for real independence.

I was trying to construct a thought experiment to show that probabilistic independence does not have to mean causal independence. For example, consider the mutually disjoint, exhaustive events:

• $A$ : it is not raining
• $B$ : the grass is not green
• $C$ : it is raining and the grass is green

I was trying to assign probabilities: $P(A) := p, P(B) := q, P(C) = 1 - p - q $ in such a nifty way as to make $A^c$ (it is raining) and $B^c$ (the grass is green) independent. We would have: $$ P(A^c \cap B^c ) = P(C) = 1-p-q $$ And from our desired independence: $$ P(A^c \cap B^c ) = P(A^c)P(B^c) = (1-P(A))(1-P(B)) = (1 - p)(1 - q) $$ Which implies that: $$ 1-p-q = (1 - p)(1 - q) $$ However, this happens only if either $p=0$ or $q=0$, in which case there is no reason to speak about the events as having any causality at all.

Is there some intuitive, snappy example of what I was trying to demonstrate? I was thinking about some variable $A$ having a causal influence on $B$, but also on some third variable $C$ which has the exact opposite effect on $B$. This would mean that $A$ and $B$ are independent, but I cannot seem to find the right tools.

Answer 1

Consider an Exclusive-OR (XOR) gate which is an electronic circuit (logic gate) with two inputs $X$ and $Y$ and an output $Z$ where $X,Y,Z$ take on values in the discrete set $\{0, 1\}$. Think of these as Boolean variables (or Bernouiii random variables if you like). $Z$ is causally related to $X$ and $Y$ by the Exclusive-OR operation: $$Z = X\oplus Y = X\bar{Y} \,\vee\, \bar{X}Y$$ if you are a Booleander or $$Z = X(1-Y)+(1-X)Y= X + Y -2XY$$ if you are a Bernoullist. Be that as it may, suppose that $X$ and $Y$ are independent (meaning that $P(X=a,Y=b) = P(X=a)P(Y=b)$ for all $a,b$ in $\{0, 1\}$. Then, \begin{align}P(Z=1) &= P(X\neq Y)\\ &=P(X=1, Y=0) + P(X=0, Y=1)\\ &= P(X=1)P(Y=0) + P(X=0)P(Y=1).\end{align} Everything OK thus far? Now suppose that $P(X=1) = P(Y=1)= \frac 12$. Then it is easy to verify that $P(Z=1) = \frac 12$ also. Now, $Z$ and $X$ are very definitely causally related: the output of an XOR gate does depend on its input(s). But, the event $\{Z=1,X=1\}$ occurs if and only if the event $\{X=1, Y=0\}$ occurs and so $$P(Z=1, X=1) = P(X=1,Y=0) = \frac 14 = P(Z=1)P(X=1) = \frac 12\times \frac 12$$ showing that the causally related events $\{Z=1\}$ and $\{X=1\}$ are in fact probabilistically independent. Similarly, $\{Z=1\}$ and $\{Y=1\}$ independent, in fact, the three events $\{X=1\}$, $\{Y=1\}$, and $\{Z=1\}$ are pairwise independent but not mutually independent since $$P(X=1, Y=1, Z=1) = 0 \neq P(X=1)P(Y=1)P(Z=1) = \frac 18.$$

Thus, causal dependence need not be reflected in probabilistic dependence; it is possible to have causally dependent events be probabilistically independent. I will also say that this probabilistic independence is purely a property of the probability measure: if we take $P(X=1)$ or $P(Y=1)$ to be any number in $(0,1)$ other than the $\frac 12$ that I sneakily chose above, the probabilistic independence disappears and the causally dependent events are also probabilistically dependent.

---

Lest you think that this is an oddball example that will hardly ever be encountered in real life, consider the gold standard in statistical theory and practice: three standard normal random variables $X,Y,Z$. Now suppose that their joint density $f_{X,Y,Z}(x,y,z)$ is not $\phi(x)\phi(y)\phi(z)$ where $\phi(\cdot)$ is the standard normal density (as would be the case if $X,Y,Z$ were mutually independent standard normal random variables), but rather

$$f_{X,Y,Z}(x,y,z) = \begin{cases} 2\phi(x)\phi(y)\phi(z) & ~~~~\text{if}~ x \geq 0, y\geq 0, z \geq 0,\\ & \text{or if}~ x < 0, y < 0, z \geq 0,\\ & \text{or if}~ x < 0, y\geq 0, z < 0,\\ & \text{or if}~ x \geq 0, y< 0, z < 0,\\ 0 & \text{otherwise.} \end{cases}\tag{1}$$ Note that $X$, $Y$, and $Z$ are not a set of three jointly normal random variables (that is, they don't have a multivariate normal distribution) but it can be shown that any two of these is indeed a pair of independent standard normal random variables. For details of the verification, see the latter half of this answer of mine.

Answer 2

In causal modelling, this kind of thing is possible in cases where there are multiple causal effects, and they happen to exactly cancel each other out in a probabilistic sense. Hence, it is possible that $\mathcal{A}$ causes $\mathcal{B}$, but it also causes $\mathcal{C}$, $\mathcal{D}$ and $\mathcal{E}$, and these latter events have a negative causal effect on $\mathcal{B}$, in a way that exactly cancels out the direct causal effect from $\mathcal{A}$.

In models of probabilistic causality this kind of pathological situation is usually ruled out by a faithfulness assumption, which assumes that the probabilistic relations are "faithful" to the underlying causal structure, and do not cancel out. A basic primer on probabilistic causality and the faithfulness assumption can be found in the Stanford Encyclopaedia of Philosophy.

Answer 3

Examples can be created at will, because causation concerns truth, but probability concerns logic.

Suppose it is a fact that $A \in \mathcal{A}$ and $B \in \mathcal{B}$ and $A = x$ causes $B = y$. Now consider given information $\mathcal{I} \equiv \unicode{8220}\text{$A \in \mathcal{A}$ and $B \in \mathcal{B}$}\unicode{8221}$. Then \begin{align} \mathrm{prob}(A = a, B = b | \mathcal{I}) &= \mathrm{prob}(A = a | \mathcal{I}) \: \mathrm{prob}(B = b | \mathcal{I}) \\ &= \frac{1}{|\mathcal{A}||\mathcal{B}|} \end{align}

because independence is the maximum entropy distribution consistent with $\mathcal{I}$.

The events are then logically independent, given $\mathcal{I}$, despite being causally dependent.

It makes no sense to speak of events being logically independent in the absence of any given assumptions: logic requires assumptions. Causes, on the other hand, exist independently of our assumptions.

Ideas about causation, of course, are themselves logical, and quite distinct from causes themselves. So if we seek to compare causal ideas about events with logical ideas about those events, in fact they are one and the same thing. For example, if we have $\mathcal{I} \equiv \unicode{8220} \text{$A \in \mathcal{A}$ and $B \in \mathcal{B}$ and $A = x$ causes $B = y$}\unicode{8221}$, then \begin{align} \mathrm{prob}(A = a, B = b | \mathcal{I}) &= \mathrm{prob}(B = b | A = a, \mathcal{I}) \: \mathrm{prob}(A = a | \mathcal{I}) \\ &= \frac{1}{|\mathcal{A}|} \begin{cases} \delta_{b y} & A = x \\ \frac{1}{|\mathcal{B}|} & A \neq x \end{cases} \end{align}

whereupon the logic expresses the causal idea.

Answer 4

Depends on what you mean by probability. If you are a frequentist, then conditional probability P(A|B) means something akin to how much B causes A. If you are Bayesian, P(A|B) only measures the logical connection between B and A. To sum up, for a Bayesian, the answer to your question is yes, for a frequentist, it is no. For ones, who are a mix of both, it is a tricky question.