# How to choose sample size from probability density for computing mutual information based on continuous variables

I need to compute mutual information gain based two continuous variables $$X$$ and $$Y$$

$$I(X|Y) = \int_X\int_Y p_{x.y}(x,y) \log(\frac{p_{x.y}(x,y)}{p_{x}(x)p_{y}(y)})$$.

I have used Kernel Density Estimation (KDE) for generating 2D density $$f_{x.y}(x,y)$$. So I have matrix $$m \times n$$.

The question is, how should I choose parameter $$m$$ and $$n$$. Generating 2d probability density using KDE is very time consuming so I have to find minimal $$m$$ and $$n$$ but I don't want to lose a lot of information. Is there any general rule which can ensure that the probability density is well sampled?

• There is a rule of thumb that is often used with binned estimators of MI (i.e. estimators in which the probabilities are replaced with histograms). It states that the number of samples should be approximately 5 times the number of bins chosen to discretize the $p(x,y)$. I don't remember where the rule originated (I think from the bias-correction theories), but it is used in practice. Maybe it is useful for your case as well. Also, beware that your estimates will be biased. – Cesare Aug 26 '19 at 8:38