When I was reading the original paper of t-SNE, I had an question whether or not we can apply KL divergence to the discrete probability distributions on different domains.
In the paper, they measure the dissimilarity between two discrete (conditional) distributions on high dimensional domains and low dimensional domains by KL divergence.
However, according to the wikipedia Kullback–Leibler divergence entry, KL divergence for discrete probability distribution is defined on the same probability space. This implies the same sample space must be used for the probability distribution.
Can we apply KL divergence to the probability distributions on different domains?