8
$\begingroup$

I am skeptical about the notion that if mutual information between two random variables is non-zero (existence) this doesn't imply the existence of correlation between them BUT existence of correlation does imply the existence of mutual information. i-e

$$ \rho \implies I(X,Y)$$

Is it correct in general?

Improve this question
$\endgroup$
2
  • $\begingroup$ Yes, it is correct. Correlation can identify linear (or even monotonic) relations, whereas mutual information is not limited to such types of relations. $\endgroup$
    – George
    Commented Jul 31, 2015 at 9:16
  • 5
    $\begingroup$ We must understand your use of the word "existence" as meaning "is nonzero," for otherwise these statements are wrong. (The mutual information always exists but the correlation might not.) This relationship becomes more apparent when stated as the contrapositive: when the mutual information is zero, the correlation must be zero. $\endgroup$
    – whuber ♦
    Commented Jul 31, 2015 at 12:15

1 Answer 1

Reset to default
9
$\begingroup$

Mutual information is zero if and only if $p(x,y) = p(x) p(y)$ and this condition implies that correlation is zero. So, if correlation is non-zero, then mutual information need to be non-zero.

Improve this answer
$\endgroup$
3
  • $\begingroup$ don't you mean "if correlation is zero, mutual information need not be zero"? $\endgroup$
    – develarist
    Commented Oct 28, 2020 at 21:53
  • 1
    $\begingroup$ @develarist No, if there is nonzero correlation, there is dependence between the marginal variables, therefore mutual information. "Need not be" means that, while it is not definite, nothing has ruled it out, either. Having nonzero correlation rules out the possibility of zero mutual information. $\endgroup$
    – Dave
    Commented Oct 28, 2020 at 22:06
  • $\begingroup$ I think users are more concerned of drawing false conclusions regarding the presence of interrelationships between variables if correlation is found to be zero, which is where mutual information comes to the rescue. Non-zero correlation and non-zero mutual information is meanwhile an obvious corollary $\endgroup$
    – develarist
    Commented Oct 28, 2020 at 22:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.