I read a paper which shows the value of normalized mutual information for two random variables is around 0.1 to 0.2 and it says so these two variables are statistical significantly correlated. I don't know why? How big the normalized mutual information is so that the variables are correlated?

  • 2
    $\begingroup$ "Statistical significance" is not a concept that applies to random variables: it applies to hypothesis tests concerning data. What data are involved here? Exactly what paper are you reading? $\endgroup$ – whuber Oct 30 '14 at 18:43
  • $\begingroup$ @whuber thanks! you are right, I think it's hypothesis test to check if two series of samples are correlated or not. The paper is maxlittle.net/publications/prd_updrs2hy.pdf and the mutual information results are shown at table 1. $\endgroup$ – Andong Zhan Oct 30 '14 at 19:13

Here's a good comparison point. The mutual information between two variables distributed according to a bivariate normal is

$$-\frac{1}{2} \log (1-\rho^2)$$

So if your mutual information is $I$, you can think of


as a generalised $R^2$

  • $\begingroup$ Presumably those two variables have identical (Normal) distributions, right? $\endgroup$ – whuber Oct 30 '14 at 19:32
  • $\begingroup$ The formula gives the mutual informaiton as a function of correlate for any bivariate normal distribution, regardless of the covariance matrix. Of course, in your case, the data may have a different joint distribution, but this should give you at least a point of comparison. $\endgroup$ – Arthur B. Oct 30 '14 at 19:42
  • $\begingroup$ Another useful comparison would be a bivariate Bernoulli. $\endgroup$ – leonbloy Oct 30 '14 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.