# Mutual Information and curse of dimensionality

Is the estimation of Shannon's mutual information from data subject to the curse of dimensionality? And if yes why?

• Do you have a specific problem in mind? I believe the answer is yes, because estimating the probability distribution of a random variable in high dimensional space is subject to the curse of dimensionality. Does it make sense? – AndreaL Dec 9 '20 at 13:26
• No. I am just considering the general problem of MI estimation. I know that there are different techniques. Maybe some are not subject to the curse of dimensionality or maybe they all are. But, if they are, it is not clear to me why. – Cesare Dec 9 '20 at 14:09

So, why are simple approaches subject to the curse of dimensionality? It's because estimating the mutual information involves an estimate of the probability density distribution $$P(X, Y)$$ and of the marginal distributions $$P(X)$$ and $$P(Y)$$.