I read a paper which shows the value of normalized mutual information for two random variables is around 0.1 to 0.2 and it says so these two variables are statistical significantly correlated. I don't know why? How big the normalized mutual information is so that the variables are correlated?
2
$\begingroup$
$\endgroup$
-
2$\begingroup$ "Statistical significance" is not a concept that applies to random variables: it applies to hypothesis tests concerning data. What data are involved here? Exactly what paper are you reading? $\endgroup$ – whuber♦ Oct 30 '14 at 18:43
-
$\begingroup$ @whuber thanks! you are right, I think it's hypothesis test to check if two series of samples are correlated or not. The paper is maxlittle.net/publications/prd_updrs2hy.pdf and the mutual information results are shown at table 1. $\endgroup$ – Andong Zhan Oct 30 '14 at 19:13
2
$\begingroup$
$\endgroup$
Here's a good comparison point. The mutual information between two variables distributed according to a bivariate normal is
$$-\frac{1}{2} \log (1-\rho^2)$$
So if your mutual information is $I$, you can think of
$$1-e^{-2I}$$
as a generalised $R^2$
-
$\begingroup$ Presumably those two variables have identical (Normal) distributions, right? $\endgroup$ – whuber♦ Oct 30 '14 at 19:32
-
$\begingroup$ The formula gives the mutual informaiton as a function of correlate for any bivariate normal distribution, regardless of the covariance matrix. Of course, in your case, the data may have a different joint distribution, but this should give you at least a point of comparison. $\endgroup$ – Arthur B. Oct 30 '14 at 19:42
-
$\begingroup$ Another useful comparison would be a bivariate Bernoulli. $\endgroup$ – leonbloy Oct 30 '14 at 19:50
