I am skeptical about the notion that if mutual information between two random variables is non-zero (existence) this doesn't imply the existence of correlation between them BUT existence of correlation does imply the existence of mutual information. i-e

$$ \rho \implies I(X,Y)$$

Is it correct in general ?

  • $\begingroup$ Yes, it is correct. Correlation can identify linear (or even monotonic) relations, whereas mutual information is not limited to such types of relations. $\endgroup$ – George Jul 31 '15 at 9:16
  • 4
    $\begingroup$ We must understand your use of the word "existence" as meaning "is nonzero," for otherwise these statements are wrong. (The mutual information always exists but the correlation might not.) This relationship becomes more apparent when stated as the contrapositive: when the mutual information is zero, the correlation must be zero. $\endgroup$ – whuber Jul 31 '15 at 12:15

Mutual information is zero if and only if $p(x,y) = p(x) p(y)$ and this condition implies that correlation is zero. So, if correlation is non-zero, then mutual information need to be non-zero.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.