15
$\begingroup$

my question particularly applies to network reconstruction

$\endgroup$
25
$\begingroup$

Correlation measures the linear relationship (Pearson's correlation) or monotonic relationship (Spearman's correlation) between two variables, X and Y.

Mutual information is more general and measures the reduction of uncertainty in Y after observing X. It is the KL distance between the joint density and the product of the individual densities. So MI can measure non-monotonic relationships and other more complicated relationships.

$\endgroup$
  • 3
    $\begingroup$ Correlation is not necessarily linear - Spearman's rho relies on the monotonic function, and yet, we refer to it as a "correlation coefficient", not "mutual information coefficient". And for a good reason: it provides an information about association between two variables. Mutual information, redundant information, mutual variance, correlation - these terms are so similar, and this question refers to network reconstruction, so I guess that we ended up in the wrong area with right terminology. This is quite specific question... $\endgroup$ – aL3xa Jul 31 '10 at 2:26
  • 1
    $\begingroup$ Good point. I've edited my answer to include monotonic relationships. I don't know anything about network reconstruction. $\endgroup$ – Rob Hyndman Jul 31 '10 at 3:04
4
$\begingroup$

To add to Rob's answer ... with respect to reverse engineering a network, MI may be preferred over correlation when you want to extract causal rather than associative links in your network. Correlation networks are purely associative. But for MI, you need more data and computing power.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy