Return to Answer

The absolute value of this area, i.e. $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ is known as the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), generally thought of within the field of optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance."

It's not entirely obvious at first that this distance corresponds to the difference in area between CDFs. But imagine doing this for two point mass distributions, one at $x$ and one at $x'$; the area between their CDFs is a rectangle, with area $1 \times \lvert x - x' \rvert$, exactly the amount that you need to move the "dirt." You can then envision doing the same thing for a collection of point masses, getting a series of rectangles that you add up. When you go to continuous distributions in the limit, you get the integral written above, and it should hopefully make sense that these are the same thing.

The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance. This relation has this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.

The absolute value of this area is $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ which note – at least for continuous distributions – is exactly equal to $$\int_{x=-\infty}^\infty \lvert F^{-1}(x) - G^{-1}(x)\rvert \,\mathrm{d}x.$$ In one dimension, the latter is the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), most commonly defined based on optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance."

It's not entirely obvious at first that this distance corresponds to the difference in area between CDFs. But imagine doing this for two point mass distributions, one at $x$ and one at $x'$; the area between their CDFs is a rectangle, with area $1 \times \lvert x - x' \rvert$, exactly the amount that you need to move the "dirt." You can then envision doing the same thing for a collection of point masses, getting a series of rectangles that you add up. When you go to continuous distributions in the limit, you get the integral written above, and it should hopefully make sense that these are the same thing.

The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance. This relation has this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.

The absolute value of this area, i.e. $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ is known as the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), generally thought of within the field of optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance." The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance. This relation has this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.

The absolute value of this area, i.e. $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ is known as the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), generally thought of within the field of optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance."

It's not entirely obvious at first that this distance corresponds to the difference in area between CDFs. But imagine doing this for two point mass distributions, one at $x$ and one at $x'$; the area between their CDFs is a rectangle, with area $1 \times \lvert x - x' \rvert$, exactly the amount that you need to move the "dirt." You can then envision doing the same thing for a collection of point masses, getting a series of rectangles that you add up. When you go to continuous distributions in the limit, you get the integral written above, and it should hopefully make sense that these are the same thing.

The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance. This relation has this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.

The integral of this area, i.e. $$\int_x \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ is known as the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), generally thought of within the field of optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance." The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance, which this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.

The absolute value of this area, i.e. $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ is known as the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), generally thought of within the field of optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance." The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance. This relation has this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.