the conditional mutual between three sets of mutually exclusive variables, X, Y, and Z, is defined as follows.

$I(X,Y|Z) = \sum_{xyz} P(x,y,z) \log \frac{P(z)P(x,y,z)}{P(x,z)P(y,z)}$

my questions concern the $\log$ of the ratio of the probability products.

  1. if $P(z)$ or $P(x,y,z)$ is 0, then $\log(0)$ is undefined.
  2. if $P(x,z)$ or $P(y,z)$ is 0, then $\log(\infty)$ is undefined.

how do i deal with these 2 situations? the approach can be very flexible. for example, i thought about ignoring the inner sums where such conditions occur, but is this correct or reasonable?

any help is appreciated.


1 Answer 1


Ignoring the terms where this happens is the correct thing to do. You can justify this by noting that in each case you've outlined, no matter what happens inside the $\log$ you will have $P(x,y,z) = 0$. You can see this by applying the Frechet inequalities, namely that $P(A,B) \le \min\{P(A), P(B)\}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.