1
$\begingroup$

Let's suppose we have 4 random variables X,Y,Z and T and that the following equations hold about the entropy: $$H(T|X)=H(T)$$ $$H(T|X,Y)=0$$ $$H(T|Y)=H(T)$$ $$H(Y|Z)=0$$ $$H(T|Z)=0$$

I want to prove the following inequality: $$I(X;Z)\geq H(T)$$

What I have done: \begin{eqnarray*} I(X;Z)\geq H(T) &\Leftrightarrow & H(X)-H(X|Z)\geq H(T)\\ &\Leftrightarrow & H(X)-H(X|Z)\geq H(T|X)\\ &\Leftrightarrow & H(X)-H(X|Z)\geq H(X,T)-H(X)\\ &\Leftrightarrow & 2H(X)\geq H(X,T)+H(X|Z)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (1)\\ \end{eqnarray*}

Also, the following properties generally hold for entropy: $$H(X)\geq H(X|Z)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (2)$$ $$H(X)+H(T)\geq H(X,T)\ \ \ \ (3)$$

Adding $(2)$ and $(3)$ side-by-side we get: $$2H(X)+H(T)\geq H(X,T)+H(X|Z)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (4)$$

So if $H(T)=0$, then $(1)$ is always true, but I am stuck here.

$\endgroup$

1 Answer 1

1
$\begingroup$

$H(Y|Z)=0$ and $H(T|Z)=0 \Rightarrow H(T+Y|Z)=H(T|Z)+H(Y|Z)-I(T;Y|Z)=0$

Then, we can use the fact that if $H(A/B)=0$ then for any $C$ , $I(B;C) \geq I(A;C)$

(intuitive if one considers that B entirely determines A)

proof (not very elegant...) :

$I(B;C)=H(B)+H(C)-H(B+C)=H(A+B)-H(A|B) +H(C)-H(A+B+C)+H(A/B+C)=H(A+B)+H(C)-H(A+B+C)=I(A+B;C) \geq I(A;C)$

So, we have that : $I(X;Z)\geq I(X;Y+T)$

and we can proceed :

$I(X;Z)\geq I(X;Y+T)=H(X)+H(Y+T)-H(X+Y+T)=H(X)+H(Y+T)-H(X+Y)-H(T/X+Y)=H(X)+H(Y)+H(T/Y)-H(X+Y)=H(X)+H(Y)+H(T/Y)-H(X)-H(Y)+I(X;Y)=H(T)+I(X;Y) \geq H(T)$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.