I read a paper which shows the value of normalized mutual information for two random variables is around 0.1 to 0.2 and it says so these two variables are statistical significantly correlated. I don't know why? How big the normalized mutual information is so that the variables are correlated?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ "Statistical significance" is not a concept that applies to random variables: it applies to hypothesis tests concerning data. What data are involved here? Exactly what paper are you reading? $\endgroup$– whuber ♦Oct 30, 2014 at 18:43
-
$\begingroup$ @whuber thanks! you are right, I think it's hypothesis test to check if two series of samples are correlated or not. The paper is maxlittle.net/publications/prd_updrs2hy.pdf and the mutual information results are shown at table 1. $\endgroup$– Andong ZhanOct 30, 2014 at 19:13
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Here's a good comparison point. The mutual information between two variables distributed according to a bivariate normal is
$$-\frac{1}{2} \log (1-\rho^2)$$
So if your mutual information is $I$, you can think of
$$1-e^{-2I}$$
as a generalised $R^2$
-
$\begingroup$ Presumably those two variables have identical (Normal) distributions, right? $\endgroup$– whuber ♦Oct 30, 2014 at 19:32
-
$\begingroup$ The formula gives the mutual informaiton as a function of correlate for any bivariate normal distribution, regardless of the covariance matrix. Of course, in your case, the data may have a different joint distribution, but this should give you at least a point of comparison. $\endgroup$ Oct 30, 2014 at 19:42
-
$\begingroup$ Another useful comparison would be a bivariate Bernoulli. $\endgroup$– leonbloyOct 30, 2014 at 19:50
-
$\begingroup$ is there a source article from where $1-e^{-2I}$ comes from discussing its usage? $\endgroup$ Dec 15, 2020 at 11:40