0
$\begingroup$

Let $\phi(x)$ be the standard normal probability density function and $f(x)$ be the Cauchy probability density function.

How can I calculate the Kullback-Leibler divergences between $\phi$ and $f$. This is

$$KL1 = \int f(x) \log \frac{f(x)}{\phi(x)}dx,$$ and $$KL2 = \int \phi(x) \log \frac{\phi(x)}{f(x)}dx?$$

Numerically, I have found that $KL2=0.259245$ and $KL1 = \infty$. Is this correct? Is $KL2<\infty$ and $KL1 = \infty$?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ The limiting value of the integrand of $KL1$ as $x\to \pm \infty$ is nonzero, because up to a normalizing constant of $1/\pi$ it behaves like $x^{-2}\log(x^{-2}/\exp(-x^2/2))=x^{-2}(-2\log(x) + x^2/2)\approx 1/2.$ Thus its integral must diverge. The integrand of $KL2$ is a multiple of $\phi,$ which goes to zero so rapidly its integral must be small in size, and certainly finite. $\endgroup$
    – whuber ♦
    Feb 5, 2022 at 0:48
  • 1
    $\begingroup$ This is answered by user Xi'an at stats.stackexchange.com/questions/351947/… and also used as an example at stats.stackexchange.com/questions/188903/… $\endgroup$
    – kjetil b halvorsen ♦
    Feb 5, 2022 at 0:49

0

Reset to default