# Kullback–Leibler divergence between normal distribution and improper distribution

Suppose that I have a a gaussian distribution $$p(x;\mu,\sigma^{2})=N(x;\mu,\sigma^{2})$$ and an improper distribution $$p(x)\propto 1, x\in \mathbb{R}.$$

I want to calculate the probabilistic distance between those two distributions, for that I use the Kullback–Leibler divergence.

$$KL(p(x;\mu,\sigma^{2}),p(x)) = \int p(x;\mu,\sigma^{2})\frac{p(x;\mu,\sigma^{2})}{p(x)}$$

Since, the $$p(x)$$ gives positive density to all the real line we have that $$p(x;\mu,\sigma^{2})$$ is absolutely continuous to the $$p(x)$$. So, we can move forward with the calculation of $$KL$$.

Then the $$KL(p(x;\mu,\sigma^{2}),p(x))= log(\frac{1}{2\pi \sigma^{2}})+\sqrt{2\pi}|\sigma|^{3}$$

Is this calculation correct?? I went through the link KL divergence between gaussian and uniform distribution and it made me wonder if my calculations are right.

• You could view the improper distribution as a limit of Normal distributions with increasing variances. Apply the formula at stats.stackexchange.com/a/7449/919.
– whuber
Sep 21, 2022 at 16:32
• @whuber So, if I do that I'll take the limit $\sigma_{2} \rightarrow \infty$ then the $KL=\infty$? Sep 21, 2022 at 16:52

$$KL(p(x;\mu,\sigma^{2}),p(x)) = \int p(x;\mu,\sigma^{2}) \log \left(\frac{p(x;\mu,\sigma^{2})}{p(x)}\right) dx$$
A problem with the improper prior $$p(x) \propto 1$$ is that the constant approaches zero and can not be eliminated. So the divergence will be infinite.