I have two signals s1 and s2, sampled 170 times each.
x = 0.2:0.2:34; s1 = rand(size(x)); s2 = randn(size(x));
The calculation of the MI (mutual information) between two discrete variables requires knowledge of their marginal probability distribution functions and their joint probability distribution.
I am estimating each signal's marginal distribution using this Kernel Density Estimator.
I am estimating the joint pdf using this 2D Kernel Density Estimator.
I created a function which takes as input the original signals, their marginal pdfs, and their joint pdf, and computes the Mutual Information.
Unfortunately, I seem to have some hug bug in this function, which I can't figure out. The MI should always be a positive number, but I am getting complex and/or negative numbers!
The function I wrote for computing the MI is shown below.
function mi = computeMI(s1, s2, pdf1, xmesh1, pdf2, xmesh2, pdf_joint, X, Y) N = size(s1, 2); p_i = zeros(1, N); p_j = zeros(1, N); for i=1:N p_i(i) = interp1(xmesh1, pdf1, s1(i)); p_j(i) = interp1(xmesh2, pdf2, s2(i)); end; mi = 0; p_ij = zeros(N, N); for j=1:N for i=1:N p_ij(i, j) = interp2(X, Y, pdf_joint, s1(i), s2(j)); delta_mi = p_ij(i,j) * (log2(p_ij(i,j) / (p_i(i) * p_j(j)))); mi = mi + delta_mi; end; end;
Thank you very much for your help.