My goal is to check if two variables $X$ and $Y$ are conditionally independent given $Z$.

For simplicity, let's assume the joint distribution is multivariate normal. In this case, we can compute partial correlation of X and Y given Z is by regressing $X \sim Z$ (with residuals $r_X$), regressing $Y \sim Z$ (with residuals $r_Y$) and computing the correlation between the residuals $r_X$ and $r_Y$. Then, conditional independence boils down to testing if this correlation is 0.

However, another way that's seemingly intuitive (at least to me), is to use the interpretation of conditional independence to test whether "knowing $Y$ helps predict $X$ any better than knowing $Z$."

That is, I can regress $X \sim Z$ (with residuals $r_X$) and regress $X \sim Y + Z$ (with residuals $r_{Y,Z}$) and test whether $r_X \ne r_{Y,Z}$ (using some appropriate statistical test or bootstrapping the distribution of the residuals).

Now, my questions are:

  • Is the second method even right?
  • If yes, what are the pros/cons of using the second method instead of the first?
  • If no, could you tell me why it's wrong?
  • $\begingroup$ I don't think your first procedure is correct unless you assume $(X,Y)\,|\,Z$ has a bivariate normal distribution. In the second procedure, exactly what "appropriate statistical test" would you use, given that $r_X$ and $r_{Y,Z}$ appear not to be independent? $\endgroup$
    – whuber
    Commented Aug 19, 2015 at 15:38
  • $\begingroup$ Yes, you are right. I meant to say "let's assume the variables are all jointly normally distributed." Let me fix the question. In the second procedure, I could bootstrap the distributions of $r_X$ and $r_{Y,Z}$ and see if the distributions are different from each other at some confidence level, perhaps using a two-sample KS test. $\endgroup$
    – Vimal
    Commented Aug 19, 2015 at 15:48
  • $\begingroup$ I don't see how that KS test would apply. Why should $r_X$ and $r_{Y,Z}$ have identical distributions under the null hypothesis? (In fact, they won't.) $\endgroup$
    – whuber
    Commented Aug 19, 2015 at 15:50
  • 4
    $\begingroup$ If you worked it out, can you please post that as an answer? $\endgroup$ Commented Dec 9, 2018 at 16:47
  • 2
    $\begingroup$ @kjetilbhalvorsen - sorry I just saw this message. You can check Appendix B in this paper for a proof: arxiv.org/abs/1903.08132. $\endgroup$
    – Vimal
    Commented Jun 18, 2019 at 17:43


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.