7
$\begingroup$

Two events $A,B$ are independent when $P(A \cap B ) = P(A)P(B)$ I am trying to drill into this definition and to try to reconcile it with our intuitive idea of independence in the real world. I feel that the equation can be achieved by accident, without any grounds for real independence.

I was trying to construct a thought experiment to show that probabilistic independence does not have to mean causal independence. For example, consider the mutually disjoint, exhaustive events:

  • $A$ : it is not raining
  • $B$ : the grass is not green
  • $C$ : it is raining and the grass is green

I was trying to assign probabilities: $P(A) := p, P(B) := q, P(C) = 1 - p - q $ in such a nifty way as to make $A^c$ (it is raining) and $B^c$ (the grass is green) independent. We would have: $$ P(A^c \cap B^c ) = P(C) = 1-p-q $$ And from our desired independence: $$ P(A^c \cap B^c ) = P(A^c)P(B^c) = (1-P(A))(1-P(B)) = (1 - p)(1 - q) $$ Which implies that: $$ 1-p-q = (1 - p)(1 - q) $$ However, this happens only if either $p=0$ or $q=0$, in which case there is no reason to speak about the events as having any causality at all.

Is there some intuitive, snappy example of what I was trying to demonstrate? I was thinking about some variable $A$ having a causal influence on $B$, but also on some third variable $C$ which has the exact opposite effect on $B$. This would mean that $A$ and $B$ are independent, but I cannot seem to find the right tools.

$\endgroup$
  • $\begingroup$ Your calculations are wrong - A and B are not mutually disjoint! $\endgroup$ – Zahava Kor Jun 22 '17 at 15:35
  • $\begingroup$ @ZahavaKor Thank you for your comment, however I never said that the grass is green only if it rains. Anyway, the whole example is incorrect and that is why I am asking this question. I just wanted to share my thought processes so far. Do you have some good example? $\endgroup$ – Martin Drozdik Jun 22 '17 at 15:43
  • 2
    $\begingroup$ The definition of probabilistic independence can be expressed in conditional probability terms as P(B/A)=P(B), which means that knowing that A has happened does not change the probability of B happening. How do you expect to find a counter-example? This is very unprobable (pun intended). $\endgroup$ – Zahava Kor Jun 22 '17 at 16:18
  • $\begingroup$ It's not clear what you mean by "causal influence". It's not (to my knowledge) a probabilistic concept and hence it's unclear how it should fit in the theory. But clearly functional dependence implies probabilistic dependence: if $X$ is a random variable and $Y = f(X)$, then $X$ and $Y$ are independent if and only if $f$ is a constant function. I'd expect the same out of any meaningful definition of causal dependence. $\endgroup$ – Olivier Feb 5 '18 at 23:49
  • $\begingroup$ @ZahavaKor yupps, I just noticed what you mean by "not mutually disjoint". Sorry, my mistake. $\endgroup$ – Martin Drozdik Jul 25 '18 at 11:39
3
$\begingroup$

Consider an Exclusive-OR (XOR) gate which is an electronic circuit (logic gate) with two inputs $X$ and $Y$ and an output $Z$ where $X,Y,Z$ take on values in the discrete set $\{0, 1\}$. Think of these as Boolean variables (or Bernouiii random variables if you like). $Z$ is causally related to $X$ and $Y$ by the Exclusive-OR operation: $$Z = X\oplus Y = X\bar{Y} \,\vee\, \bar{X}Y$$ if you are a Booleander or $$Z = X(1-Y)+(1-X)Y= X + Y -2XY$$ if you are a Bernoullist. Be that as it may, suppose that $X$ and $Y$ are independent (meaning that $P(X=a,Y=b) = P(X=a)P(Y=b)$ for all $a,b$ in $\{0, 1\}$. Then, \begin{align}P(Z=1) &= P(X\neq Y)\\ &=P(X=1, Y=0) + P(X=0, Y=1)\\ &= P(X=1)P(Y=0) + P(X=0)P(Y=1).\end{align} Everything OK thus far? Now suppose that $P(X=1) = P(Y=1)= \frac 12$. Then it is easy to verify that $P(Z=1) = \frac 12$ also. Now, $Z$ and $X$ are very definitely causally related: the output of an XOR gate does depend on its input(s). But, the event $\{Z=1,X=1\}$ occurs if and only if the event $\{X=1, Y=0\}$ occurs and so $$P(Z=1, X=1) = P(X=1,Y=0) = \frac 14 = P(Z=1)P(X=1) = \frac 12\times \frac 12$$ showing that the causally related events $\{Z=1\}$ and $\{X=1\}$ are in fact probabilistically independent. Similarly, $\{Z=1\}$ and $\{Y=1\}$ independent, in fact, the three events $\{X=1\}$, $\{Y=1\}$, and $\{Z=1\}$ are pairwise independent but not mutually independent since $$P(X=1, Y=1, Z=1) = 0 \neq P(X=1)P(Y=1)P(Z=1) = \frac 18.$$

Thus, causal dependence need not be reflected in probabilistic dependence; it is possible to have causally dependent events be probabilistically independent. I will also say that this probabilistic independence is purely a property of the probability measure: if we take $P(X=1)$ or $P(Y=1)$ to be any number in $(0,1)$ other than the $\frac 12$ that I sneakily chose above, the probabilistic independence disappears and the causally dependent events are also probabilistically dependent.


Lest you think that this is an oddball example that will hardly ever be encountered in real life, consider the gold standard in statistical theory and practice: three standard normal random variables $X,Y,Z$. Now suppose that their joint density $f_{X,Y,Z}(x,y,z)$ is not $\phi(x)\phi(y)\phi(z)$ where $\phi(\cdot)$ is the standard normal density (as would be the case if $X,Y,Z$ were mutually independent standard normal random variables), but rather

$$f_{X,Y,Z}(x,y,z) = \begin{cases} 2\phi(x)\phi(y)\phi(z) & ~~~~\text{if}~ x \geq 0, y\geq 0, z \geq 0,\\ & \text{or if}~ x < 0, y < 0, z \geq 0,\\ & \text{or if}~ x < 0, y\geq 0, z < 0,\\ & \text{or if}~ x \geq 0, y< 0, z < 0,\\ 0 & \text{otherwise.} \end{cases}\tag{1}$$ Note that $X$, $Y$, and $Z$ are not a set of three jointly normal random variables (that is, they don't have a multivariate normal distribution) but it can be shown that any two of these is indeed a pair of independent standard normal random variables. For details of the verification, see the latter half of this answer of mine.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you! This is a great answer and exactly what I was looking for. Awesome! $\endgroup$ – Martin Drozdik Feb 18 '19 at 15:25
  • 1
    $\begingroup$ @MartinDrozdik But the answer that you have already accepted rules out such examples as pathological and due to a lack of a "faithfulness assumption". I am not sure what exactly is "unfaithful" in what I have written in my answer above (choosing Bernoulli random variables to have parameter $\frac 12$ which is the default in the absence on any preconceived notions is faithlessness?), but then I am not a philosopher. $\endgroup$ – Dilip Sarwate Feb 18 '19 at 15:48
  • $\begingroup$ An illuminating example. The independence between $X$ and $Z$ even persists regardless of the value of $P(X = 1)$, provided $P(Y = 1) = \frac{1}{2}$. Independence also holds for $P(X = 1) = 1$ and $P(X = 1) = 0$, regardless of $P(Y = 1)$. The independence in this example, however, depends upon the selected cause not being sufficient. But even sufficient causes can be probabilistically independent of their effects. $\endgroup$ – CarbonFlambe--Reinstate Monica Feb 19 '19 at 12:44
5
$\begingroup$

In causal modelling, this kind of thing is possible in cases where there are multiple causal effects, and they happen to exactly cancel each other out in a probabilistic sense. Hence, it is possible that $\mathcal{A}$ causes $\mathcal{B}$, but it also causes $\mathcal{C}$, $\mathcal{D}$ and $\mathcal{E}$, and these latter events have a negative causal effect on $\mathcal{B}$, in a way that exactly cancels out the direct causal effect from $\mathcal{A}$.

In models of probabilistic causality this kind of pathological situation is usually ruled out by a faithfulness assumption, which assumes that the probabilistic relations are "faithful" to the underlying causal structure, and do not cancel out. A basic primer on probabilistic causality and the faithfulness assumption can be found in the Stanford Encyclopaedia of Philosophy.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Examples can be created at will, because causation concerns truth, but probability concerns logic.

Suppose it is a fact that $A \in \mathcal{A}$ and $B \in \mathcal{B}$ and $A = x$ causes $B = y$. Now consider given information $\mathcal{I} \equiv \unicode{8220}\text{$A \in \mathcal{A}$ and $B \in \mathcal{B}$}\unicode{8221}$. Then \begin{align} \mathrm{prob}(A = a, B = b | \mathcal{I}) &= \mathrm{prob}(A = a | \mathcal{I}) \: \mathrm{prob}(B = b | \mathcal{I}) \\ &= \frac{1}{|\mathcal{A}||\mathcal{B}|} \end{align}

because independence is the maximum entropy distribution consistent with $\mathcal{I}$.

The events are then logically independent, given $\mathcal{I}$, despite being causally dependent.

It makes no sense to speak of events being logically independent in the absence of any given assumptions: logic requires assumptions. Causes, on the other hand, exist independently of our assumptions.

Ideas about causation, of course, are themselves logical, and quite distinct from causes themselves. So if we seek to compare causal ideas about events with logical ideas about those events, in fact they are one and the same thing. For example, if we have $\mathcal{I} \equiv \unicode{8220} \text{$A \in \mathcal{A}$ and $B \in \mathcal{B}$ and $A = x$ causes $B = y$}\unicode{8221}$, then \begin{align} \mathrm{prob}(A = a, B = b | \mathcal{I}) &= \mathrm{prob}(B = b | A = a, \mathcal{I}) \: \mathrm{prob}(A = a | \mathcal{I}) \\ &= \frac{1}{|\mathcal{A}|} \begin{cases} \delta_{b y} & A = x \\ \frac{1}{|\mathcal{B}|} & A \neq x \end{cases} \end{align}

whereupon the logic expresses the causal idea.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.