According to this webpage, the bias correction term when estimating $I(X;Y)$ for discrete random variables $X,Y$ is $\sim \textrm{df}(X,Y)/N$ where $\textrm{df}(X,Y)$ is the degrees of freedom of the joint distribution of $X$ and $Y$ such that $\textrm{df}(X,Y)= \textrm{df}(X)\textrm{df}(Y)$ and $N$ is number of (paired) observations. If I instead want to estimate $I(A;B)$ using joint distributions $A = \{A_0,A_1,...\}$ and $B = \{B_0,B_1,...\}$, would $\textrm{df}(A,B) = \textrm{df}(A_0)\textrm{df}(A_1)...\textrm{df}(B_0)\textrm{df}(B_1)...$? This seems correct intuitively as uncertainty should increase with the addition of variables (and therefore mutual information should decrease).

I understand this estimation is only reasonable for $\textrm{df}(A,B) \ll N$, which is not the case for the sequence abundance data I want to eventually work with, but I'm writing a python module and I want to include this estimation technique among many.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.