I have posted a related Question based on a paper, Ma, Jian, and Zengqi Sun. "Mutual information is copula entropy." Tsinghua Science & Technology 16.1 (2011): 51-54.
In the paper, they say that mutual information $I(x)$ is related to copula entropy $H_c(x)$ via $I(x) = -H_c(x)$.
However, the sign of entropy is supposed to be non-negative, as is the sign of mutual information. This leads me to conclude that mutual information and copula entropy are zero. If mutual information is zero, then all variables are unrelated, so something here is borked.
In the linked question, ArnoV commented that differential entropy can be either positive or negative (or zero), and the R
package by the paper authors appears to be for continuous variables (it does not like having tied values). Fine. I even have heard that copula theory gets a bit weird for discrete marginal variables (copulae are not unique), so perhaps we should restrict ourselves to the continuous case. Okay...
Then $H_c(x)\in\mathbb{R}$; entropy can be any real value. This then means that mutual information can be any real value.
But mutual information is not negative! This leads right back to both mutual information and copula entropy having to be zero. Contradiction! $\Rightarrow\Leftarrow$
The result in the paper is elegant and makes a lot of sense to me. If the copula describes the relationship between marginal variables and the mutual information quantifies the degree to which the marginal variables are dependent, they should be related. Yet the equation they give seems to result in math being broken.
What's going on? What am I missing?