What is meant by divergence in statistics?

I have learned about the Intuition on the Kullback-Leibler (KL) Divergence as how much a model distribution function differs from the theoretical/true distribution of the data.

The two most important divergences are the relative entropy (Kullback–Leibler divergence, KL divergence), which is central to information theory and statistics, and the squared Euclidean distance (SED).

Is divergence a distance/measure of the distance?

What is the cleanest, easiest way to explain to someone the concept of divergence? Could anyone explain in detail about divergence in layman's terms?

• while 'convergence' is pretty unambiguous, 'divergence' can indicate the opposite of 'convergence' or a completely different thing. Apr 20, 2021 at 14:53
• We need some context to disambiguate the intended meanings.
– whuber
Apr 20, 2021 at 15:03

Informally, people sometimes describe divergences as measuring the "distance" between probability distributions. This risks confusion with formal distance metrics, which must satisfy some extra requirements. In addition to the requirements above, a distance metric must also be symmetric: $$D(a,b) = D(b,a)$$. And, it must satisfy the triangle inequality: $$D(a,c) \le D(a,b) + D(b,c)$$. As a side note, divergences are defined specifically on probability distributions, whereas distance metrics can be defined on other types of objects too.