2
$\begingroup$

I came across a book where the author uses the following property of mutual information:

Let $X$,$Y$,$Z$ be arbitrary discrete random variables and let $W$ be an indicator random variable.

$$ (1)\ \ I[ X : Y \mid Z ] = Pr(W=0) I[ X : Y \mid Z, W=0 ] + Pr(W=1) I[ X : Y \mid Z, W=1 ]\ $$

I don't understand why this property holds in general. To show this I was thinking to proceed as follows: \begin{align} I[ X : Y \mid Z ] &= E_z[ I[ X : Y \mid Z = z ]] \\ &= E_w[ E_z[ I[ X: Y \mid Z = z]\ |\ W=w ] ] \\ &= Pr(W=0)E_z[ I[ X: Y \mid Z = z]\ |\ W=0 ] \\ &+ Pr(W=1)E_z[ I[ X: Y \mid Z = z]\ |\ W=1 ]. \end{align} where the second line follows by the law of total expectation. However, this does not seem to be the right approach since it's not clear to me that $$ E_z[ I[ X: Y \mid Z = z]\ |\ W=w ] = I[ X : Y \mid Z, W=w ]$$ holds.

What is the right way to show (1)?

$\endgroup$
1
  • 1
    $\begingroup$ can you share the source and page/slide number? $\endgroup$
    – gunes
    Feb 16, 2020 at 16:52

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.