If we know A is independent of B, why isn't P(A|B,C) = P(A|C) necessarily true?

Question

Let's say we know that A is independent of B, or mathematically:

$$P(A|B) = P(A)$$

Then how come we can't say the following is necessarily true: $$P(A|B,C) = P(A|C)$$

If the outcome of B doesn't have an effect on the outcome of $A$, then why would the outcome of $B$ AND $C$ have an effect on $A$ that is different than the effect of the outcome of just $C$?

Is there perhaps a simple counterexample to illustrate this? For example, let's say $A$ and $B$ are the event of picking an ace of spades from a deck of cards (each has their own deck of cards).

Answer 1

My favourite example is a chessboard: if you pick a point uniformly then the row, column, and colour are pairwise independent.

Suppose A is colour, B is row, and C is column. Then P(A="white"|B)=0.5 and P(A="white"|C)=0.5 for any value of B, but P(A="white"|B,C) is either 0 or 1 for any values of B and C.

Answer 2

Imagine $A$ and $B$ are independent Bernoulli random variables with success probability 1/2, and let $C = AB + (1 - A)(1 - B)$ (if you like, $A$ is the event that a coin lands heads, $B$ is the event that another coin lands heads, and $C$ is the event that both coins land the same side up). Then $P(A = 1 \mid C = 1) = 1/2$ but $P(A = 1 \mid B = 1, C = 1) = 1$.

A bit more generally, a common situation where this is not true is when $(A, B, C)$ form a v-structure, i.e., where $A \rightarrow C \leftarrow B$, where $A$ and $B$ are marginally independent but where $B$ can provide information about $A$ if both cause $C$.

Answer 3

Let's continue your example with two decks of cards, supposing that one card is chosen randomly from each deck. You have proposed that $A$ is the event that the ace of spades is chosen from the first deck and that $B$ is the event that the ace of spades is chosen from the second deck.

Now let $C$ be the event that the two cards from the two decks are of different colors. Then $$P(A|C) = P(A) = 1/52$$ but $$P(A|B,C) = 0$$ because $A$, $B$ and $C$ cannot simultaneously be true.

Note that any two of $A$, $B$ and $C$ are independent but the three events are dependent when taken together.

Answer 4

There are two situations how this can happen.

Collider bias

A common way how the situation can occur is collider bias.

An example is in the image below. In this example, intelligence and appearance are statistically unrelated (there's not even an effect in this case as with the coin example below). But conditional on the sum of intelligence and appearance the two become correlated.

Invariant effect

With collider bias both A and B have an effect on C. In some special cases, the situation may also appear when B has a causal effect on A, but it does not appear as a statistical dependency.

The underlying reason why this counterintuitive situation can happen is because: When we speak about statistical independence between A and B then it doesn't necessarily mean that A and B have no functional/causal relationship.

Let's say that we flip two fair coins $B$ and $C$, and assign values $-1$ and $1$ to the sides of the coins. Let's define $A := B \times C$.

 B   C   A := B*C
 1   1   1
 1  -1  -1
-1   1  -1
-1  -1   1

Now $B$ does influence the outcome $A$ but $A$ is not statistically dependent on $B$ alone. Namely we have that $P(A = 1) = P(A = -1) = 0.5$ independent of the value of $B$.

Variants of this case occur when $B$ is a variable that influences $A$ while leaving the distribution of $A$ unchanged. So this case happens when $B$ has an effect on $A$ that leaves the distribution of $A$ invariant. E.g. it happens when $B$ switches labels of a fair coin or dice.

This seems like a contrived case, but it is also a bit about a particular interpretation of probability. When we flip a coin, does it matter whether we start with heads up or tails up? Statistically speaking there might not be a relationship. But in the underlying physical process the starting position of the coin has (according to theory) an influence on the position of the coin after the flip. The situation may, after all, be considered deterministic; but we describe it as probabilistic because we have no complete knowledge about the begin state of the entire system and all the parameters that are involved in the outcome.

Answer 5

I quote Joseph Blitzstein's Introduction to Probability (2019 2 edn), p. 65.

Example 2.5.11 (Independence doesn't imply conditional independence).

My friends Alice and Bob are the only two people who ever call me on the phone. Each day, they decide independently whether to call me that day. Let A be the event that Alice calls me next Friday and B be the event that Bob calls me next Friday. Assume A and B are unconditionally independent with P(A) > 0 and P(B) > 0.

However, given that I receive exactly one call next Friday, A and B are no longer independent: the call is from Alice if and only if it is not from Bob. In other words, letting C be the event that I receive exactly one call next Friday, $P(B|C) > 0$ while $P(B|A,C) = 0$, so A and B are not conditionally independent given C.

Answer 6

As @Sextus Empiricus (the ancient skeptic!) said, collider bias phenomenon in epidemiology is a great exemplar of why marginal independence of two random variables does not necessarily imply their conditional independence, given a third random variable.

Sackett (1978)'s example is a classic for the collider bias. In his paper, $A =$ having locomotor disease, and $B = $ having respiratory disease. Let $C$ denote the event that a person is hospitalized (for whatever reason). He observed that among hospitalized patients, there was a strong association between $A$ and $B$, whereas there was no association between $A$ and $B$ in the overall population (mix of both $C$ and $\bar C$). If $C$ denotes hospitalization, then we see that $P(A|B,C) \ne P(A|C)$ and $P(B|A,C) \ne P(B|C)$, even though $P(A|B) = P(B)$, this is because both $A$ and $B$ increase the probability of $C$. Hospitalization, $C$, is the collider, being a common effect of $A$ and $B$.