# What is the meaning of independent but conditionally dependent (Gaussian case)?

I am trying to understand what it means for random variables to be (unconditionally) independent but conditionally dependent. Let $(X,Y,Z)$ be a set of three random variables. I understand the usual example of coin flips: $X$ is Bernoulli representing one coin, $Y$ is Bernoulli representing another coin, and $Z = X+ Y$. Here, knowledge of the values of $Z$ and $X$ help in the determination of the value of $Y$ (likewise, $Z$ and $Y$ for $X$).

However, I am confused about Gaussian random variables. Let the set $(X,Y,Z)$ now be a set of jointly Gaussian random variables. Let $Z=X+Y+N(0,1)$. Usually, independence and conditional independence for Gaussians are assessed by correlation and partial correlation, respectively.

$X$ and $Y$ are independent iff the correlation coefficient between $X$ and $Y$ is $r=0$. Similarly, $X$ and $Y$ are independent given $Z$ iff the partial correlation coefficient between $X$ and $Y$ is $r \not = 0$. What exactly is happening that is making the partial correlation coefficient become non-zero here? In the partial correlation test, we regress out $Z$ from $X$ and $Y$ separately, so I don't understand how $r$ becomes non-zero between $X$ and $Y$...

First of all, don't ever write things like $Z = X+Y+N(0,1)$, not in public as on this forum, not on your exams, and not even in your notebook or private diary: in fact, don't even think of $N(0,1)$ as being the same as a standard normal random variable. Second, disabuse yourself of the notion that "$X$ and $Y$ are independent iff the correlation coefficient between $X$ and $Y$ is $r=0$." This assertion is not always true, not even when $X$ and $Y$ are restricted to be normal random variables. (It is true if we restrict $X$ and $Y$ to be jointly normal random variables; the presence of that adjective jointly is crucial here).

Turning to your question, you seem to be starting from three independent normal random variables $X, Y, W$ and then considering the three variables $X, Y$, and $Z = X+Y+W$. Since the vector $(X,Y,Z)$ is obtained via a linear transformation from the jointly normal vector $(X,Y,W)$, $X, Y$, and $Z$ also have a jointly normal distribution. Note, though, that while (by assumption) $X, Y, W$ are mutually independent random variables, $X, Y$, and $Z$ are not mutually independent. In fact, both $X$ and $Y$ have nonzero correlation with $Z$. Specifically,

\begin{align} \operatorname{cov}(X,Z) &= \operatorname{cov}(X,X+Y+W) = \operatorname{cov}(X,X) = \operatorname{var}(X)\\ \operatorname{cov}(Y,Z) &= \operatorname{cov}(Y,X+Y+W) = \operatorname{cov}(Y,Y) = \operatorname{var}(Y) \end{align}

Also, $$\mu_Z = \mu_X + \mu_Y + \mu_W, \quad \text{and} \quad\operatorname{var}(Z) = \operatorname{var}(X) + \operatorname{var}(Y) + \operatorname{var}(W).$$

Turning to the joint distribution of $X$ and $Y$ conditioned on $Z = z$, note first that this is a bivariate normal distribution. It is a standard result (see e.g. Wikipedia) that the mean vector of this bivariate distribution is

$$\left[\begin{matrix}\hat{\mu}_X\\ \\ \hat{\mu}_Y \end{matrix}\right] = \left[\begin{matrix}{\mu}_X + \left.\left.\frac{\operatorname{var}(X)}{\operatorname{var}(Z)}\right(z - \mu_Z\right)\\ {\mu}_Y + \left.\left.\frac{\operatorname{var}(Y)}{\operatorname{var}(Z)}\right(z - \mu_Z\right)\end{matrix}\right]$$ while the covariance matrix is $$\left[\begin{matrix}\operatorname{var}(X) & 0\\ 0 & \operatorname{var}(Y)\end{matrix}\right] - \frac{1}{\operatorname{var}(Z)}\left[\begin{matrix}(\operatorname{var}(X))^2 & \operatorname{var}(X)\operatorname{var}(Y)\\ \operatorname{var}(X)\operatorname{var}(Y) & (\operatorname{var}(Y))^2\end{matrix}\right].$$ Notice that the covariance matrix is not a diagonal matrix, that is,

$X$ and $Y$ are independent normal random variables (and thus jointly normal random variables too), but, conditioned on the value of $Z = X+Y+W$, they are conditionally dependent (jointly normal) random variables.

Why is this so? Well, broadly speaking, knowing the value of $Z$ tells us a little bit about the values of $X$ and $Y$, and what is still unknown about $X$ and $Y$ is thus linked through this shared information. Instead of being independent, they are conditionally dependent. Think of $Z$ as a noisy observation of $X+Y$ where the "noise" is $W$. If the observation of $X+Y$were completely noiseless, then it should be obvious that $X$ and $Y$, even though they are independent, cannot possibly be conditionally independent given that $X+Y = z$ because $Y$ must necessarily equal $z-X$, that is, the value of $X$ determines the value of $Y$. Even for noisy observations, we should be skeptical of assertions or insistences that $X$ and $Y$ are conditionally independent given that $X+Y+W = z$, and indeed the math shows that $X$ and $Y$ are conditionally dependent random variables given the value of a noisy observation $X+Y+W$ of their sum.