If you draw samples of unknown distribution, how can you measure the KL-divergence between the unknown distribution and a gaussian distribution with zero mean and unit variance N(0,1)?
Can we use the moments measured from the drawn samples? Or how can we approach this?
Additionally, it would be interesting if the method is differentiable.
Some related work I found:
In [1], the methods are implemented to compare unknown distributions and you only have the samples from both distributions.
In [2], the authors state, "It has been known for some time that in the case of the Gaussian distribution, matching the first two moments of the original density yields the optimal approximation in terms of minimizing the KLD." and they prove that also true for some hyperspherical probability distributions, namely the von Mises–Fisher and the Watson distribution.
References:
[1] KL-Divergence Estimators, https://github.com/nhartland/KL-divergence-estimators
[2] Kurz, Gerhard, Florian Pfaff, and Uwe D. Hanebeck. "Kullback–Leibler Divergence and Moment Matching for Hyperspherical Probability Distributions", https://www.researchgate.net/publication/305449272_Kullback-Leibler_Divergence_and_Moment_Matching_for_Hyperspherical_Probability_Distributions