# Mutual Information between multi-dimensional and single dimensional Variables

I would like to estimate MI between two variables X and Y of shape (nXd) and (nX1) respectively. As X is multi-dimensional (dimension = d) and Y is of dimsnension=1, is it a correct approach of the finding MI between each dimension of X against Y and adding each dimension's MI to get final MI? Sample example:

X = rand[N,d]
Y = rand[N,]

for i=1 to d:
mi_total += calculate_MI( X[:,i] , Y) #X[:,i] i_th dimension of X

# X[:,i].shape = (N,1) = Y.shape


Any help and correction is very much appreciated and Thanks in advance :)

Theoretically speaking, the following quantities are not equal in general, i.e. $$I(\mathbf{X};Y)\neq \sum_{i=1}^dI(X_i;Y)$$ Because, we are dropping out any dependence relationship. For example, if $$\mathbf{X}=[X_1,X_1]$$ (i.e. $$X_2=X_1$$), then the mutual information between $$\mathbf{X}$$ and $$Y$$ would be $$I(X_1;Y)$$, not $$I(X_1;Y)+I(X_2;Y)=2I(X_1;Y)$$. However, if the random variables inside $$\mathbf{X}$$ are independent, you can sum their individual MIs, as you've done in your code. I couldn't capture yours, but in many coding languages rand is assumed to create independent samples, in which you can really sum MIs.
• Thank you. X in my case is a dataset with d dimension, so I reckon it is safe assuming that each dimension is independent of each other and also the reason for using rand. I think my doubt is cleared :) – Vigneswaran C Jun 18 '19 at 16:08