Suppose we have three random variables, $X$, $Y_1$, and $e$ (for error). Variable $e$ is independent of $X$ and $Y_1$, but $X$ and $Y_1$ are dependent. Further suppose we construct a new mixture variable $Y_2$ which is $Y_1$ observed under error $e$, assuming the additive functional form $$Y_2= Y_1+e.$$

Now I suspect that $$ P(X|Y_1,Y_2) \ne P(X|Y_2).$$ I am looking for ways to prove this statement.

Edit (additional information): In the special case I am interested in $X$ is discrete (Bernoulli or Binomial) and $Y_1$ and $e$ are normally distributed.

| cite | improve this question | | | | |
  • $\begingroup$ Note I gave one more information to make the question more specific for my case ($X$ is binary, $Y_1$ and $e$ are bivariate normal). It would be interesting to discuss my question in general or in this particular situation. The solution suggested by @Chaconne concerns the situation when $X$ is normal and was given before I added the additional information. $\endgroup$ – tomka May 30 '16 at 10:36

We can use a graphical model to answer questions like this. I just made this in MS paint so it doesn't look very nice, but the idea should be clear. I also renamed your $E$ as $e$ just so there's no confusion with the expectation operator if that gets involved.

If I've understood your question correctly, we have that $X$ and $Y_1$ are dependent, so there is an edge connecting them. We also have that $Y_2$ is correlated with $Y_1$ and $e$ so there are those two edges. There are no other dependencies. I used a directed graphical model because it seems like you want to view $Y_2$ as being "caused" by $Y_1$ and $e$.

You really want to know if $X \perp Y_1 \vert Y_2$. Looking at the graphical model, we can say that this is not the case. Cover up the node representing $Y_2$: there is still a pathway between $X$ and $Y_1$. That means that there is still a relationship between $X$ and $Y_1$ even when $Y_2$ is known.

This can all be made much more rigorous if you are looking to make this a proof.

enter image description here

Update: here's a counterexample using first principles.

Let's say $(X, Y, e) \sim \mathcal N_3(\vec 0, \Sigma)$ where $$ \Sigma = \left[ {\begin{array}{ccc} 1 & \rho & 0\\ \rho & 1 & 0 \\ 0&0&1 \end{array} } \right] $$

so that $X$ and $Y$ are correlated and both are independent of $e$.

We know that for $Z \sim \mathcal N(0, \Omega)$ $$ f_Z(z) \propto \exp(-\frac{1}{2} z^t \Omega^{-1} z). $$

I used Wolfram Alpha to evaluate this with our particular covariance matrix so that $$ f_{(X,Y,e)}(x, y, e) \propto \exp\left[-\frac{1}{2}\left( \frac{x^2}{1-\rho^2} + 2 \frac{xy\rho}{1-\rho^2} + \frac{y^2}{1-\rho^2} + e^2 \right)\right] $$

Now let $(U, V, W) = (X, Y, Y + e)$. This is a linear transformation so we don't need to worry about a Jacobian, and we simply get $$ f_{(U,V,W)}(u, v, w) = f_{(X,Y,e)}(u, v, w-v) $$

which means that $$ f_{(U,V,W)}(u,v,w) \propto \exp\left[-\frac{1}{2}\left( \frac{u^2}{1-\rho^2} + 2 \frac{uv\rho}{1-\rho^2} + \frac{v^2}{1-\rho^2} + (w-v)^2 \right)\right]. $$

$X \perp Y_1 \vert Y_2$ in this example is equivalent to checking if $U \perp V \vert W$, which means that we would need to be able to factor $f_{(U,V,W)}(u,v,w)$ so that there are no terms involving both $u$ and $v$. Clearly when $\rho \neq 0$ this is impossible, and therefore we have a counterexample to the claim.

| cite | improve this answer | | | | |
  • $\begingroup$ Thanks, I was already thinking along these lines too (hence the tag on graphical models I added). I would be interested in proving this beyond the figure (which makes perfect sense in itself but is not exactly statistical without knowledge of Pearl's book). $\endgroup$ – tomka May 27 '16 at 13:33
  • $\begingroup$ The approach you deleted looked interesting, what was wrong about it? $\endgroup$ – tomka May 27 '16 at 14:39
  • $\begingroup$ I think there was a mistake in the covariance matrix after the transformation and I wanted to fix it offline. $\endgroup$ – jld May 27 '16 at 14:40
  • $\begingroup$ @ThomasKlausch Ok I redid it. Hopefully it's all correct now $\endgroup$ – jld May 27 '16 at 15:09
  • 1
    $\begingroup$ My understanding was that you were trying to show that it is not the case that $X \perp Y_1 \vert Y_2$. A counterexample is sufficient to show this. I certainly did not characterize all random variables such that this is true, but I have proven that it cannot be that for all random variables with the dependencies that you've described $X \perp Y_1 \vert Y_2$. Maybe I misunderstood what you were trying to do here? $\endgroup$ – jld May 27 '16 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.