mutual information between discrete random variables and continuous variables How does one compute mutual information (MI) between discrete random variables and continuous random variables ?
It's easy to compute MI for dataset with only discrete random variables. For dataset with continuous random variables, MI can be computed by discretising the continuous random variable.
However, when the dataset contains both discrete and continuous random variables, how do I compute the MI between these two types of random variables ?
Any credible recommended papers?
 A: Note that there is a difference between defining and calculating the MI of two random variables $X$, $Y$ whose joint distribution you know and actually estimating the MI from data $\left(X_i, Y_i\right)_{i = 1, \dots, n}$.
In any case see e.g. [1] for an estimator of MI for continuous variables without discretization, [2] and [3] for extensions of this estimator to discrete-continuous settings. [3] also has Definition 2.1 where MI is defined in terms of the Radon-Nikodym derivative of the joint distribution $\mathbf P^{X,Y}$ w.r.t. the product measure $\mathbf P^X \otimes \mathbf P^X$ which is valid for all types of random variables (essentially MI is the KL-divergence between these two measures). [3] also gives some guarantees, e.g. that the estimated MI is consistent for $n \to \infty$ (under some conditions).  Off the top of my hat I unfortunately do not know about more rigorous papers (though I don't doubt that they exist).
[1]A. Kraskov, H. Stoegbauer, and P. Grassberger, ‘Estimating Mutual Information’, Phys. Rev. E, vol. 69, no. 6, p. 066138, Jun. 2004, doi: 10.1103/PhysRevE.69.066138.
[2]B. C. Ross, ‘Mutual Information between Discrete and Continuous Data Sets’, PLoS ONE, vol. 9, no. 2, p. e87357, Feb. 2014, doi: 10.1371/journal.pone.0087357.
[3]W. Gao, S. Kannan, S. Oh, and P. Viswanath, ‘Estimating Mutual Information for Discrete-Continuous Mixtures’, in Advances in Neural Information Processing Systems, 2017, vol. 30. Accessed: May 24, 2022. [Online]. Available: https://proceedings.neurips.cc/paper/2017/hash/ef72d53990bc4805684c9b61fa64a102-Abstract.html
