Why is KL divergence non-negative?
From the perspective of information theory, I have such an intuitive understanding:
Say there are two ensembles $A$ and $B$ which are composed of the same set of elements labeled by $x$. $p(x)$ and $q(x)$ are different probability distributions over ensemble $A$ and $B$ respectively.
From the perspective of information theory, $\log_{2}(P(x))$ is the least amount of bits that required for recording an element $x$ for ensemble $A$. So that the expectation $$\sum_{x \in ensemble}-p(x)\ln(p(x))$$ can be interpreted as at least how many bits that we need for recording an element in $A$ on average.
Since this formula puts a lower bound on the bits that we need on average, so that for a different ensemble $B$ which brings about a different probability distribution $q(x)$, the bound that it gives for each element $x$ will surely not bit that is given by $p(x)$, which means taking the expectation,
$$\sum_{x\in ensemble}-p(x)\ln(q(x))$$
this average length will surely be greater than the former one, which leads to
$$\sum_{x\in ensemble }p(x)\frac{\ln(p(x))}{\ln(q(x))} > 0$$
I don't put $\ge$ here since $p(x)$ and $q(x)$ are different.
This is my intuitive understanding, is there a purely mathematical way of proving KL divergence is non-negative? The problem can be stated as:
Given $p(x)$ and $q(x)$ are both positive over real line, and $\int_{-\infty}^{+\infty}p(x)dx = 1$, $\int_{-\infty}^{+\infty}q(x)dx = 1$. Prove $$\int_{-\infty}^{+\infty}p(x)\ln\frac{p(x)}{q(x)}$$ is non-negative.
How can this be proved? Or can this be proved without extra conditions?