Does independence imply conditional independence?

Question

If two or more variables A, B, C, etc. are jointly mutually independent of one another, does this imply that that they are also conditionally independent given some set of conditioning variables X, Y, Z, etc.?

If you test variables A, B, C,, etc., and the hypothesis of independence is not rejected, are you generally safe in assuming that these variables will also survive a test of conditional independence given X, Y, Z, etc.? Is it safe enough that you do not need to bother running the latter test? Why or why not?

Answer 1

No.

Consider three boolean variables: A, B, X where X and A are i.i.d. Bernoulli with probabilty 0.5, while B = X $\oplus$ A (that is, B is equal to the xor of X and A).

It's easy to show that B is also Bernoulli distributed with probabilty 0.5, and A and B are mutually independent, though obviously they aren't conditionally independent given X.

Answer 2

There are four possible two-edge trails from X to Y via Z as shown in this figure:

Source: Probabilistic Graphical Models Principles and Techniques (2009, The MIT Press)

These four relations represent all possible structures in a Bayes Graph(we can extend the graph to contain more than three nodes respectively). We can learn from the book that if Z is given, X and Y are independent in the first three relations(this also meets the requirement in your case: X and Y are mutually independent if we imagine that X and Y contain multiple nodes). But in (d), where three nodes form a v-structure, if Z is given X and Y become dependent.

Answer 3

Two events $A$ and $B$ that are stochastically independent (meaning that $P(A\cap B) = P(A)P(B)$ holds need not be conditionally independent given that some event $C$ occurred, that is, it need not be the case that $P((A\cap B) \mid C) = P(A\mid C)P(B\mid C)$ which is the definition of conditional independence of $A$ and $B$ conditioned on $C$. For example, if we take $C$ to be the event $A\cup B$, then since $A\cap B$, $A$, and $B$ all are subsets of $A\cup B$, we have that \begin{align} P((A\cap B) \mid (A\cup B)) &= \frac{P(A\cap B)}{P(A\cup B)}\\ P(A \mid (A\cup B)) &= \frac{P(A)}{P(A\cup B)}\\ P(B \mid (A\cup B)) &= \frac{P(B)}{P(A\cup B)}\\ \end{align} and it should be obvious that

$$\text{since we have assumed that}~P(A\cap B) = P(A)P(B)~ \text{holds,}\\ \text{it cannot possibly be true that}~ \frac{P(A\cap B)}{P(A\cup B)}=\frac{P(A)}{P(A\cup B)}\times\frac{P(B)}{P(A\cup B)}~\text{also holds}.$$

Nitpickers getting ready to object "But it holds when $P(A\cup B)$ equals $1$, doesn't it??" are reminded that $P(A\cup B) = 1-P(A^c\cap B^c)$ cannot equal $1$ for independent events $A$ and $B$ except in the trivial case when at least one of $A$ and $B$ is an event of probability $1$. $\big($Note that $P(A\cup B)=1$ implies that $P(A^c\cap B^c) = 0$ but by independence, $P(A^c\cap B^c)=P(A^c)P(B^c)$ equals $0$ and so at least one of $A^c$ and $B^c$ has probability $0$ and its complement has probability $1.\big)$ So

Independent events need not be conditionally independent.

But of course there exist conditioning events $C$ such that independent events $A$ and $B$ are also conditionally independent given $C$. Trivially, if $A,B,C$ are stochastically mutually independent events, then $A$ and $B$ are also conditionally independent events conditioned on the occurrence of $C$.

---

Turning to random variables, if $X$ and $Y$ are independent Bernoulli$\left(\frac 12\right)$ random variables and $Z$ is a Bernoulli random variable that has value $1$ if and only if $X \neq Y$, then $X$ and $Z$ are a pair of independent Bernoulli$\left(\frac 12\right)$ random variables as are $Y$ and $Z$ a pair of independent Bernoulli$\left(\frac 12\right)$ random variables. But, the conditional joint pmf of $X$ and $Y$ given $Z$ does not factor into the product of the conditional marginal pdfs of $X$ and $Y$ showing that $X $and $Y$ are not conditionally independent random variables given $Z$. (This is essentially Antonio's example). But note also that the independence of $X$ and $Z$ and also of $Y$ and $Z$ is stochastic independence that disappears if we assume that $X$ and $Y$ are independent random Bernoulli random variables with parameter $p \notin \{0,\frac 12, 1\}$. So, if one is estimating probabilities of random variables and using these estimates to determine whether it is reasonable to assume stochastic independence or not, then such an approach if fraught with peril. While one might not reject the hypothesis that $p = \frac 12$, not rejecting the null is not the same as a whole-hearted acceptance of the null as the gospel truth.

See also this earlier answer of mine on this topic.

Answer 4

It depends on your definition of independence. I have seen at least one textbook that used the term "stocastic indepedence" or something like that to refer to the case where 2 variables were independent, and still conditionally independent when conditioned on other variables. Other textbooks seem to imply this added independence when using the term "independent".

A simple example of 2 variables that are independent without being stocastically independent are the (x,y,z) triplets: (1,2,3); (1,3,2); (2,1,3); (2,3,1); (3,1,2); (3,2,1); (1,1,1); (2,2,2); and (3,3,3). Given any value of x (or y or z) the probabilies for y (or z) are 1/3 for each of the values 1,2,3. But if you know x and y (or any pair) then you know the value of z.

Answer 5

Here is another counter-example. Suppose X, Y and Z are jointly Gaussian with covariance matrix:

\begin{bmatrix} 1 & 0 & \rho \\ 0 & 1 & \rho \\ \rho & \rho & 1 \end{bmatrix}

for some non-zero $\rho$.

Since $X$ and $Y$ are jointly Gaussian and have 0 correlation, they are independent. However, their conditional joint distribution given $Z$ is normal with covariance matrix:

\begin{bmatrix} 1 - \rho^2 & -\rho^2 \\ -\rho^2 & 1-\rho^2 \end{bmatrix}

Since $\rho \ne 0$, $X$ and $Y$ are not independent.