I have two datasets $D1$ and $D2$ in two different feature spaces $\mathcal{X}_{1} \in \Re^{m}$ and $\mathcal{X}_{2} \in \Re^{n}$. Further assume that the datasets have different number of data points.

I also do not have any idea about the distribution that each of the above datasets are coming from. Can I still compute the KL divergence between the two datasets ?

I came across the fact that unless we know the distribution of the datasets (or fit a distribution to the datasets), it is not possible for us to compute the KL divergence.

Any comments ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.