1
$\begingroup$

According to this webpage, the bias correction term when estimating $I(X;Y)$ for discrete random variables $X,Y$ is $\sim \textrm{df}(X,Y)/N$ where $\textrm{df}(X,Y)$ is the degrees of freedom of the joint distribution of $X$ and $Y$ such that $\textrm{df}(X,Y)= \textrm{df}(X)\textrm{df}(Y)$ and $N$ is number of (paired) observations. If I instead want to estimate $I(A;B)$ using joint distributions $A = \{A_0,A_1,...\}$ and $B = \{B_0,B_1,...\}$, would $\textrm{df}(A,B) = \textrm{df}(A_0)\textrm{df}(A_1)...\textrm{df}(B_0)\textrm{df}(B_1)...$? This seems correct intuitively as uncertainty should increase with the addition of variables (and therefore mutual information should decrease).

I understand this estimation is only reasonable for $\textrm{df}(A,B) \ll N$, which is not the case for the sequence abundance data I want to eventually work with, but I'm writing a python module and I want to include this estimation technique among many.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.