**The Kullback-Leibler divergence is unbounded.** Indeed, since there is no lower bound on the $q(i)$'s, there is no upper bound on the $p(i)/q(i)$'s. For instance, the Kullback-Leibler divergence between a Normal $N(\mu_1,\sigma_1)$ and a Normal $N(\mu_2,\sigma_2)$ is $$\frac{1}{2\sigma_1^{2}}(\mu_1-\mu_2)^2$$which is clearly unbounded. [Wikipedia][1] [which has been known to be wrong!] indeed states > "...a Kullback–Leibler divergence of 1 indicates that the two > distributions behave in such a different manner that the expectation > given the first distribution approaches zero." which makes no sense (expectation of which function? why 1 and not 2?) A more satisfactory explanation from the same [Wikipedia][1] page is that the Kullback–Leibler > "...can be construed as measuring the expected number of extra bits > required to code samples from P using a code optimized for Q rather > than the code optimized for P." [1]: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence