# Proof that the mutual information I(X;Y) between two random variables X and Y is 0 if and only if X and Y are independent

On the Wikipedia of mutual information it says that $$I(X;Y)=0$$ if and only if $$X$$ and $$Y$$ are independent.

I can proof that if $$X$$ and $$Y$$ are independent, then $$I(X;Y)=0$$, because $$p(x,y)=p(x)p(y)$$. But how can one prove that if $$I(X;Y)=0$$, then $$X$$ and $$Y$$ are independent?

## 1 Answer

Here's my take. In the discrete case,

$$\operatorname{I}(X;Y) = \sum_{y \in \mathcal Y} \sum_{x \in \mathcal X} {p_{(X,Y)}(x,y) \log{ \left(\frac{p_{(X,Y)}(x,y)}{p_X(x)\,p_Y(y)} \right) } }$$

So $$I(X;Y)=0$$ when, at all points, either:

1. $${p_{(X,Y)}(x,y)} = 0$$, or
2. $$\log{ \left(\frac{p_{(X,Y)}(x,y)}{p_X(x)\,p_Y(y)} \right) } \;=0$$

It's worth noting that MI is always negative or 0, and we can't get a positive logarithm at any point, because the joint probability is always a subset of the marginals, so we don't need to worry about sums cancelling each other out; just that each $$\operatorname{I}(X;Y)=0\, \forall X,Y$$. (And of course, by definition, any data point can't reduce the total amount of information.)

The first case says the joint probability is 0 in those cases. It's essentially saying that the two events can't happen together, so I think we're ok just treating these as impossible or undefined events.

The second case requires that $$\frac{p_{(X,Y)}\,(x,y)}{p_X(x)\,p_Y(y)} = 1$$, which implies $$p_{(X,Y)} = p_X(x)\,p_Y(y)$$, which is independence, for all cases where both events can happen together.

The logic is the same in the continuous case.