I found this awesome thread which shows KL divergence between two univariate Gaussians. I was wondering if the same formula worked for KL divergence b/w 2 univariate Poisson distributions.

Or should I use the general KL divergence formula and plug into it the pdf for a Poisson process: $$\int { pdf1(x)*{ log(pdf1(x)/pdf2(x))} } $$

where for Poisson is $$pdf(x) = (\lambda^x / x!)*e^{-\lambda} $$


1 Answer 1


This is fairly straight-foward. The log ratio of the densities is equal to:

$$ \log\left (\frac {f_1}{f_2}\right)=x\log\left (\frac {\lambda_1}{\lambda_2}\right)+\lambda_2-\lambda_1$$

Then you take expectation of this expression wrt $ f_1$, which simply replaces $ x $ with its expectation (in this case). So you have:

$$ D_{KL} (f_1||f_2)=\lambda_1\log\left (\frac {\lambda_1}{\lambda_2}\right)+\lambda_2-\lambda_1$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.