How can I show that two random variables are independent if their mutual information is 0? On the Wikipedia page for mutual information, it says that $I(X;Y)=0$ if and only if $X$ and $Y$ are independent.
I can proof that if $X$ and $Y$ are independent, then $I(X;Y)=0$, because $p(x,y)=p(x)p(y)$. But how can one prove that if $I(X;Y)=0$, then $X$ and $Y$ are independent?
 A: Here's my take. In the discrete case,
$$ \operatorname{I}(X;Y) = 
\sum_{y \in \mathcal Y} \sum_{x \in \mathcal X}
    {p_{(X,Y)}(x,y) \log{ \left(\frac{p_{(X,Y)}(x,y)}{p_X(x)\,p_Y(y)} \right) }
} $$
So $I(X;Y)=0$ when, at all points, either:

*

*${p_{(X,Y)}(x,y)} = 0$, or

*$\log{ \left(\frac{p_{(X,Y)}(x,y)}{p_X(x)\,p_Y(y)} \right) } \;=0$
It's worth noting that MI is always negative or 0, and we can't get a positive logarithm at any point, because the joint probability is always a subset of the marginals, so we don't need to worry about sums cancelling each other out; just that each $\operatorname{I}(X;Y)=0\, \forall X,Y$. (And of course, by definition, any data point can't reduce the total amount of information.)
The first case says the joint probability is 0 in those cases. It's essentially saying that the two events can't happen together, so I think we're ok just treating these as impossible or undefined events.
The second case requires that $\frac{p_{(X,Y)}\,(x,y)}{p_X(x)\,p_Y(y)} = 1$, which implies $p_{(X,Y)} = p_X(x)\,p_Y(y)$, which is independence, for all cases where both events can happen together.
The logic is the same in the continuous case.
