Let X,Y,Z be jointly distributed, the conditional mutual information is defined as :- enter image description here

Similarly if we have 4 (or more variables) say X,Y,Z,W then how can we determine the conditional mutual information between X and Y given Z and W. So basically, how to determine conditional mutual information between 2 variables based on multiple conditions.

  • $\begingroup$ Where is this screenshot from? $\endgroup$ Commented Jan 29, 2023 at 23:07

1 Answer 1


Define the random variable $V \triangleq (W, Z)$ and this becomes a case of the equation in your screenshot. You can then compute $\operatorname{I}(X; Y \mid V)$.

  • $\begingroup$ So you mean V is the joint probablity of the multiple conditions. In this case it will be joint probablity between W and Z. Could you please elaborate if I am getting it right. $\endgroup$
    – Abhi
    Commented May 17, 2021 at 20:27
  • $\begingroup$ Yeah, that's right. $\endgroup$ Commented May 17, 2021 at 20:29
  • $\begingroup$ Thanks. Just one clarification, since I m not that familiar, if I want to calculate the joint probablity between 2 discreate random variables, X and Y. And X has 4 events(x1,x2,x3,x4) and Y has 2 (y1,y2). So the combined joint probablity will be the summation of all the events taken in different combinations, right. Like sum of joint prob of (x1,y1); (x1,y2); (x2,y1); (x2,y2)... $\endgroup$
    – Abhi
    Commented May 18, 2021 at 10:05
  • $\begingroup$ Also, can we extend the formula of conditional mutual information shown above to include more conditions using Entropy. Like instead of Z can we write H( XWZ) + H(YWZ) + H(XYWZ) -H(WZ). Will such formula be applied ? $\endgroup$
    – Abhi
    Commented May 18, 2021 at 12:06
  • $\begingroup$ Yes, you’re right about the joint distribution. Yes, you can modify the formula like that. $\endgroup$ Commented May 18, 2021 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.