# Looking for proof of conditional dependence, when the conditioning variables are linearly related

Suppose we have three random variables, $X$, $Y_1$, and $e$ (for error). Variable $e$ is independent of $X$ and $Y_1$, but $X$ and $Y_1$ are dependent. Further suppose we construct a new mixture variable $Y_2$ which is $Y_1$ observed under error $e$, assuming the additive functional form $$Y_2= Y_1+e.$$

Now I suspect that $$P(X|Y_1,Y_2) \ne P(X|Y_2).$$ I am looking for ways to prove this statement.

Edit (additional information): In the special case I am interested in $X$ is discrete (Bernoulli or Binomial) and $Y_1$ and $e$ are normally distributed.

• Note I gave one more information to make the question more specific for my case ($X$ is binary, $Y_1$ and $e$ are bivariate normal). It would be interesting to discuss my question in general or in this particular situation. The solution suggested by @Chaconne concerns the situation when $X$ is normal and was given before I added the additional information. – tomka May 30 '16 at 10:36

We can use a graphical model to answer questions like this. I just made this in MS paint so it doesn't look very nice, but the idea should be clear. I also renamed your $E$ as $e$ just so there's no confusion with the expectation operator if that gets involved.

If I've understood your question correctly, we have that $X$ and $Y_1$ are dependent, so there is an edge connecting them. We also have that $Y_2$ is correlated with $Y_1$ and $e$ so there are those two edges. There are no other dependencies. I used a directed graphical model because it seems like you want to view $Y_2$ as being "caused" by $Y_1$ and $e$.

You really want to know if $X \perp Y_1 \vert Y_2$. Looking at the graphical model, we can say that this is not the case. Cover up the node representing $Y_2$: there is still a pathway between $X$ and $Y_1$. That means that there is still a relationship between $X$ and $Y_1$ even when $Y_2$ is known.

This can all be made much more rigorous if you are looking to make this a proof. Update: here's a counterexample using first principles.

Let's say $(X, Y, e) \sim \mathcal N_3(\vec 0, \Sigma)$ where $$\Sigma = \left[ {\begin{array}{ccc} 1 & \rho & 0\\ \rho & 1 & 0 \\ 0&0&1 \end{array} } \right]$$

so that $X$ and $Y$ are correlated and both are independent of $e$.

We know that for $Z \sim \mathcal N(0, \Omega)$ $$f_Z(z) \propto \exp(-\frac{1}{2} z^t \Omega^{-1} z).$$

I used Wolfram Alpha to evaluate this with our particular covariance matrix so that $$f_{(X,Y,e)}(x, y, e) \propto \exp\left[-\frac{1}{2}\left( \frac{x^2}{1-\rho^2} + 2 \frac{xy\rho}{1-\rho^2} + \frac{y^2}{1-\rho^2} + e^2 \right)\right]$$

Now let $(U, V, W) = (X, Y, Y + e)$. This is a linear transformation so we don't need to worry about a Jacobian, and we simply get $$f_{(U,V,W)}(u, v, w) = f_{(X,Y,e)}(u, v, w-v)$$

which means that $$f_{(U,V,W)}(u,v,w) \propto \exp\left[-\frac{1}{2}\left( \frac{u^2}{1-\rho^2} + 2 \frac{uv\rho}{1-\rho^2} + \frac{v^2}{1-\rho^2} + (w-v)^2 \right)\right].$$

$X \perp Y_1 \vert Y_2$ in this example is equivalent to checking if $U \perp V \vert W$, which means that we would need to be able to factor $f_{(U,V,W)}(u,v,w)$ so that there are no terms involving both $u$ and $v$. Clearly when $\rho \neq 0$ this is impossible, and therefore we have a counterexample to the claim.

• Thanks, I was already thinking along these lines too (hence the tag on graphical models I added). I would be interested in proving this beyond the figure (which makes perfect sense in itself but is not exactly statistical without knowledge of Pearl's book). – tomka May 27 '16 at 13:33
• The approach you deleted looked interesting, what was wrong about it? – tomka May 27 '16 at 14:39
• I think there was a mistake in the covariance matrix after the transformation and I wanted to fix it offline. – jld May 27 '16 at 14:40
• @ThomasKlausch Ok I redid it. Hopefully it's all correct now – jld May 27 '16 at 15:09
• My understanding was that you were trying to show that it is not the case that $X \perp Y_1 \vert Y_2$. A counterexample is sufficient to show this. I certainly did not characterize all random variables such that this is true, but I have proven that it cannot be that for all random variables with the dependencies that you've described $X \perp Y_1 \vert Y_2$. Maybe I misunderstood what you were trying to do here? – jld May 27 '16 at 15:24