Mutual information is not zero for independent variables and negative for weakly dependency To the best of my knowledge, mutual information (MI) is zero if and only if the variables are independent.
I have simulated copula data and computed the MI and the results are as follows:

*

*Independent copula  (MI = 0.04)

*Very weak dependency Gaussian copula (MI = -0.03)!


*

*My question is, is it acceptable to have negative MI? Is that because the relationship between mutual information and copula is negative?


*Is it acceptable to have > 0 for completely independent variables.
I used an 'infotheo` package in R.
 A: The mutual information should be always non-negative. So whatever estimator you use, say $\hat I$, I would just define a new estimator based on it say $\hat J= \max(\hat I,0)$. If $\hat I$ is always nonnegative, $\hat J = \hat I$. Otherwise
$\hat J$ would be a strictly better estimator (closer to the truth). $\hat J$ could be biased, but who cares! This is one example where you should trade-off bias for something else.
Personally, I would use an estimator based on the KL divergence definition of the mutual information which is guaranteed to be nnonegative, not based on estimating the entropies in $ H(X) + H(Y) - H(X,Y)$. The idea is to use binning, compute a contingency table (a discretized version of the joint distribution), and then compute the KL divergence between the joint distribution and the distribution with corresponding independent marginals. (There should be ways to fix the bias you get due to binning if the number of samples in each bin is very small. Otherwise this should be a good estimate in itself. Choosing the bin size is generally the tricky problem).
Regarding "Is it acceptable to have > 0 for completely independent variables", you just have to understand the "sampling error" (the inherent uncertainly caused by looking at a finite sample of a distribution). Then it would hopefully make perfect sense.
