# KL divergence between gaussian and uniform distribution

Is the KL divergence not defined because uniform has bounded support and gaussian has unbounded support?

How else can I calculate the distance of my gaussian to a 'maximum entropy' distribution if I can't use the uniform distribution?

• The distance is infinite. – Xi'an May 21 at 6:28

## 1 Answer

The KL divergence $$KL\left(P \middle\| Q\right) = \int \log \frac{d P}{d Q }dP$$ is only defined if the Radon-Nikodym derivative exists, which is when $$P$$ is absolutely continuous with respect to $$Q$$ (written $$P \ll Q$$). This means that there can't be any sets $$A$$ where $$P(A) > 0$$ and $$Q(A) = 0$$, otherwise we would be dividing by zero.

In your case, $$p$$ is the density of the uniform random variable, and $$q$$ is the density of the normal random variable (they are both dominated by the Lebesgue measure), so you could calculate $$KL\left(P \middle\| Q\right) = \int \log \frac{p(x)}{q(x)}p(x)dx,$$ but you couldn't calculate $$KL\left(Q \middle\| P\right)$$. You can calculate $$KL\left(P \middle\| Q\right)$$ because there are no sets $$A$$ such that $$\int_A p(x) dx > 0$$ and $$\int_A q(x) dx = 0$$.

• Yes, unfortunately I need KL(Q||P). j – user3180 May 21 at 4:11