Is it possible that marginally independent random variables are conditionally dependent?

Question

Suppose that $X,Y$ and $Z$ are random variables. If $X$ is independent of $Z$ and $Y$ is independent of $Z$, is it possible that $X$ is dependent on $Z$ given $Y$ and $Y$ is dependent on $Z$ given $X$? If so, what are some examples?

Answer 1

Here a simple example with three Bernoulli random variables $X,Y,Z$ with the properties that

$X$ and $Z$ are independent, $Y$ and $Z$ are independent and

• $X$ and $Z$ are conditionally dependent random variables given the value of $Y$,
• $Y$ and $Z$ are conditionally dependent random variables given the value of $X$,

Suppose that $(X,Y,Z)$ takes on the $4$ values $(0,0,0), (0,1,1), (1,0,1), (1,1,0)$ with equal probability $\frac 14$. It is easily verified that $X, Y$, and $Z$ are indeed Bernoulli random variables with parameter $\frac 12$, and that $X, Y$, and $Z$ are indeed pairwise independent random variables. (Those too lazy to carry out this verification for themselves can read some details here.) But notice that

• given that $Y=0$, $X$ equals $Z$, while given that $Y=1$, $X$ equals $1-Z$ and thus $X$ and $Z$ are conditionally dependent given $Y$.

Similarly,

• given that $X=0$, $Y$ equals $Z$ while given that $X=1$, $Y$ equals $1-Z$ and thus $Y$ and $Z$ are conditionally dependent given $X$.

For these specific random variables, it is also true that $X$ and $Y$ are independent random variables; in fact, $X,Y,Z$ are pairwise independent but not mutually independent random variables. In fact, for these specific random variables, it is also true that

• given that $Z=0$, $X$ equals $Y$ while given that $Z=1$, $X$ equals $1-Y$ and thus $X$ and $Y$ are conditionally dependent given $Z$,

which has a pleasing symmetry with the two previous bulleted points. These extra properties might not hold for other sets of random variables that satisfy the requirements laid out in the problem posed by the OP (which don't include the requirement of conditional dependence of $X$ and $Y$ given $Z$).

Answer 2

@Dilip Sarwate: have already given a good answer, the purpose of this is to show the connection with Simpson's paradox. If such examples did not exist, Simpson's paradox would have been eliminated. The easy way to see is with a simple example with artificial data. Let $X, Y, Z$ have the possible values $-1, +1$. In the two strata defined by $Z$ we have the following tables, with $X$ in rows, $Y$ in columns:

 dist[, , 1]  # Table for Z=-1:
    Y
X    -1 1
  -1  2 4
  1   4 2
 dist[, , 2]  # Table for Z=+1:
    Y
X    -1 1
  -1  4 2
  1   2 4

Observe that $X$ and $Y$ are dependent in both these conditional distributions. But all the three bivariate marginals are uniform, so we have pairwise independence:

 apply(dist, c(1, 2), sum)
    Y
X    -1 1
  -1  6 6
  1   6 6
 apply(dist, c(1, 3), sum)
    Z
X    -1 1
  -1  6 6
  1   6 6
 apply(dist, c(2, 3), sum)
    Z
Y    -1 1
  -1  6 6
  1   6 6

So we have an instance of Simpson's paradox: Conditioning shows an effect which cannot be seen in the marginal distribution. Compare with examples here: How to resolve Simpson's paradox?

Construction of example data in R:

dist <- array(c(2, 4, 4, 2, 4, 2, 2, 4),
              dim=c(2, 2, 2),
              dimnames=list(c(-1, 1),  c(-1, 1), c(-1, 1)))

names(dimnames(dist)) <- c("X",  "Y",  "Z")
```