1
$\begingroup$

The KL is given by: $D_{\mathrm{KL}}(P\|Q) = \int_X p \, \log \frac{p}{q} \, d\mu.$

The PDF of a Lognormal distribution is given by: $P = \frac 1 x \cdot \frac 1 {\sigma\sqrt{2\pi\,}} \exp\left( -\frac{(\ln x-\mu)^2}{2\sigma^2} \right).$

My Question is if it is possible to obtain an analytical solution for the KL with the given PDFs. I've searched the google and found nothing on it, except this post

Where the solution to my question is given as:

$$ D(f_i\|f_j)= \frac1{2\sigma_j^2}\left[(\mu_i-\mu_j)^2+\sigma_i^2-\sigma_j^2\right] + \ln \frac{\sigma_j}{\sigma_i}, $$ But I think this is wrong. I calculated numerically the KL for two Lognormal distributions and it does not match with this solution.

With these parameter:

$p(x)=\text{Lognormal}\left(0,1\right),~~q(x)=\text{Lognormal}\left(5,4\right)$

I get numerically:

$D_{KL_{numerical}} = 1.256$

The expression on the other hand yields:

$D_{KL_{analytical}} = 2.776$

Maybe someone else could provide more insight here because I was unable to derive an analytical solution for the KL of two Lognormal distributions and it was asked at least one time already here.

If someone has some clever tricks for how to derive it, i will try my best and, if successful, post the solution here.

EDIT:

thanks to kjetil b halvorsen who pointed out that in this thread this paper was named to contain the same solution for my problem as it was stated in the post i was mentioning originally.

Still the problem that i can't validate this solution persists. Can someone maybe confirm the given solution or give me a hint how to derive it ?

$\endgroup$
  • $\begingroup$ I think the derivation is in mast.queensu.ca/~communications/Papers/gil-msc11.pdf , one of the references to the pdf in the paper cited above. (In the text of the cited paper, it refers to the derivations as having been done in the master's thesis I cite herein.) $\endgroup$ – jbowman Dec 6 '17 at 15:53
  • 2
    $\begingroup$ the Kullback-Leibler between two log-Normals is the same as the pdf between the corresponding Normals. And your numerical example is useless if you do not provide all parameters. $\endgroup$ – Xi'an Dec 7 '17 at 19:45