# Which KL Divergence is larger D(P|Q) or D(Q|P)?

From the perspective of information theory, I understand how D(P|Q) is non-negative and why the KL divergence is asymmetric, i.e. $$D(P|Q) \neq D(Q|P)$$, given two gaussian univariate gaussian distributions.

I also know that $$D(P|Q) = ln\frac{\sigma_q}{\sigma_p} + \frac{\sigma_p^2 + (\mu_p - \mu_q)^2}{2\sigma_q^2} - \frac{1}{2}$$

My question is that if the mean of both the divergence is equal, i.e. $$\mu_p = \mu_q$$, Which of the divergence would be larger, D(P|Q) or D(Q|P)?

• Hi Inderpartap, welcome! Do you have an idea of how to check the sign of $D(P|Q) - D(Q|P)$? – Konstantin Nov 12 '19 at 10:28

It depends on the ratio of deviations. Let $$x=\sigma_p/\sigma_q$$ When $$\mu_p=\mu_q$$, $$D(P|Q)=-\ln x+x^2/2-1/2, D(Q|P)=\ln x+{1 \over 2x^2}-1/2$$ As you might guess, these two are equal when $$x=1$$ and there is no other solution because if $$x=a$$ is a solution, so as $$x=1/a$$; and for $$x>1$$, this expression is greater than $$0$$, i.e. there is no zero-crossing.
To sum up, $$D(P|Q)$$ is larger when $$\sigma_p>\sigma_q$$.