my question particularly applies to network reconstruction
2 Answers
$\begingroup$
$\endgroup$
3
Correlation measures the linear relationship (Pearson's correlation) or monotonic relationship (Spearman's correlation) between two variables, X and Y.
Mutual information is more general and measures the reduction of uncertainty in Y after observing X. It is the KL distance between the joint density and the product of the individual densities. So MI can measure non-monotonic relationships and other more complicated relationships.
answered Jul 31, 2010 at 1:08
-
3$\begingroup$ Correlation is not necessarily linear - Spearman's rho relies on the monotonic function, and yet, we refer to it as a "correlation coefficient", not "mutual information coefficient". And for a good reason: it provides an information about association between two variables. Mutual information, redundant information, mutual variance, correlation - these terms are so similar, and this question refers to network reconstruction, so I guess that we ended up in the wrong area with right terminology. This is quite specific question... $\endgroup$– aL3xaCommented Jul 31, 2010 at 2:26
-
1$\begingroup$ Good point. I've edited my answer to include monotonic relationships. I don't know anything about network reconstruction. $\endgroup$ Commented Jul 31, 2010 at 3:04
-
$\begingroup$ @RobHyndman Dear sir, can you please answer my question? I have asked the question about 15 hours ago. However, still no answer received. $\endgroup$ Commented Apr 11, 2020 at 5:32
$\begingroup$
$\endgroup$
To add to Rob's answer ... with respect to reverse engineering a network, MI may be preferred over correlation when you want to extract causal rather than associative links in your network. Correlation networks are purely associative. But for MI, you need more data and computing power.
