4
$\begingroup$

My goal is to check if two variables $X$ and $Y$ are conditionally independent given $Z$.

For simplicity, let's assume the joint distribution is multivariate normal. In this case, we can compute partial correlation of X and Y given Z is by regressing $X \sim Z$ (with residuals $r_X$), regressing $Y \sim Z$ (with residuals $r_Y$) and computing the correlation between the residuals $r_X$ and $r_Y$. Then, conditional independence boils down to testing if this correlation is 0.

However, another way that's seemingly intuitive (at least to me), is to use the interpretation of conditional independence to test whether "knowing $Y$ helps predict $X$ any better than knowing $Z$."

That is, I can regress $X \sim Z$ (with residuals $r_X$) and regress $X \sim Y + Z$ (with residuals $r_{Y,Z}$) and test whether $r_X \ne r_{Y,Z}$ (using some appropriate statistical test or bootstrapping the distribution of the residuals).

Now, my questions are:

  • Is the second method even right?
  • If yes, what are the pros/cons of using the second method instead of the first?
  • If no, could you tell me why it's wrong?
$\endgroup$
  • $\begingroup$ I don't think your first procedure is correct unless you assume $(X,Y)\,|\,Z$ has a bivariate normal distribution. In the second procedure, exactly what "appropriate statistical test" would you use, given that $r_X$ and $r_{Y,Z}$ appear not to be independent? $\endgroup$ – whuber Aug 19 '15 at 15:38
  • $\begingroup$ Yes, you are right. I meant to say "let's assume the variables are all jointly normally distributed." Let me fix the question. In the second procedure, I could bootstrap the distributions of $r_X$ and $r_{Y,Z}$ and see if the distributions are different from each other at some confidence level, perhaps using a two-sample KS test. $\endgroup$ – Vimal Aug 19 '15 at 15:48
  • $\begingroup$ I don't see how that KS test would apply. Why should $r_X$ and $r_{Y,Z}$ have identical distributions under the null hypothesis? (In fact, they won't.) $\endgroup$ – whuber Aug 19 '15 at 15:50
  • 2
    $\begingroup$ If you worked it out, can you please post that as an answer? $\endgroup$ – kjetil b halvorsen Dec 9 '18 at 16:47
  • 1
    $\begingroup$ @kjetilbhalvorsen - sorry I just saw this message. You can check Appendix B in this paper for a proof: arxiv.org/abs/1903.08132. $\endgroup$ – Vimal 2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.