# How does the triangle inequality have anything to do with relative entropy?

I am reading Elements of information theory by Thomas M. Cover, and Joy A. Thomas second edition. and on page 19, chapter 2, section 3, it says:

I am confused by the random mentioning of triangles.

Out of the blue, this sounds like a typo. I am starting to understand relative entropy and its dependence on probability mass distributions but I have not used any triangles in any of my proofs.

### Why does the triangle inequality have anything to do with entropy?

I have copied the text from the image below:

In the above deﬁnition, we use the convention that $$0\log \frac00 =0$$ and the convention (basedoncontinuity arguments) that $$0log \frac0q =0$$ and $$p\log \frac{p}0 =\infty$$. Thus, if there is any symbol $$x \in X$$ such that $$p(x) > 0$$ and $$q(x)=0$$, then $$D(p||q)=\infty$$.

We will soon show that relative entropy is always nonnegative and is zero if and only if $$p = q$$. However, it is not a true distance between distributions since it is not symmetric and does not satisfy the triangle inequality. Nonetheless, it is often useful to think of relative entropy as a “distance” between distributions.

In mathematics, in particular geometry, a distance function on a given set $$M$$ is a function $$d: M \times M \to \mathbb{R}$$, where $$\mathbb{R}$$ denotes the set of real numbers, that satisfies the following conditions:
• $$d(x,y) \ge 0$$, and $$d(x,y) = 0$$ if and only if $$x = y$$. (Distance is positive between two different points, and is zero precisely from a point to itself.)
• It is symmetric: $$d(x,y) = d(y,x)$$. (The distance between $$x$$ and $$y$$ is the same in either direction.)
• It satisfies the triangle inequality: $$d(x,z) \le d(x,y) + d(y,z)$$. (The distance between two points is the shortest distance along any path). Such a distance function is known as a metric. Together with the set, it makes up a metric space.
It just says that we do not have $$D(p||q) \le D(p||r) + D(r||q)$$ in general. In fact, it is not even symmetric.