How to measure the shift between two cumulative distribution functions (CDFs)? Specifically, in the image below, how meaningful is the shaded area? It is supposed to measure the shift between the green and the red CDF. On the x-axis, the value of the variable is given in arbitrary units, while on the y-axis, it's the cumulative density. I'm concerned that, given it's all cumulative, the differences in the lower left keep contributing to the area further up, so that there is a much stronger weight on the differences between values that are leftmost on the x-axis, where cumulation begins. Am I right? Am I wrong? If so, where is my misunderstanding?

shift area

  • 5
    $\begingroup$ The shaded (signed) area is the difference in expectations of the two distributions. There are no "weights" in any parts of the area--it's just an area. "Signed" means that areas where the red curve is to the left of the green curve are subtracted from the total. (No such areas appear in this image). $\endgroup$
    – whuber
    Commented Aug 23, 2017 at 12:42
  • 3
    $\begingroup$ Orthogonal comments, picky if you like, but in graphics details can matter a lot. 1. Difficulty distinguishing red and green is a common problem many people have; here it doesn't matter because the two curves would be distinct whatever the colours, but even for this kind of problem, red and green curves that crossed could be a puzzle. Red and blue or orange and blue are much less problematic. 2. Diagonal shading is meaningless here. Appropriate shading should just be monochrome. $\endgroup$
    – Nick Cox
    Commented Aug 23, 2017 at 13:17
  • $\begingroup$ @Nick: You're correct, I'll use red/blue in the end. $\endgroup$ Commented Aug 25, 2017 at 3:59
  • $\begingroup$ For future readers who are interested in a proof of why the signed area is the difference in expectations, a partial proof is given here under "Proof of expected survival time formula". $\endgroup$
    – mhdadk
    Commented Apr 15, 2022 at 13:20

2 Answers 2


Think about what a CDF represents in terms of probability. Let the variables on the x-axis be referred to as $x$ and y-axis values be referred to as $y$. By definition the cumulative distribution function is showing the probability that a variable is less than or equal to $x$. More specifically, if you look at $x=0$ for each curve the CDF is telling you: $P_{\text{red}}(X \leq 0) \approx 0.5$ and $P_{\text{green}}(X \leq 0) \approx 0.7$.

Your question is a little vague so I will answer it in two parts.

How meaningful is the difference at a particular point?: Let's assume $X$ represents the difference in points from an average test score (with a negative value representing below average and positive value representing above average). Let the green curve represent boys and red curve represent girls. Now, $P_{\text{red}}(X \leq 0) \approx 0.5$ and $P_{\text{green}}(X \leq 0) \approx 0.7$ tells us that the probability of a boy scoring below average is higher than the probability of a girl scoring below average. If we look at the CDF as whole (green always above red) this suggests in your sample population, girls score higher than boys. Whether or not this result is statistically significant is yet to be determined.

How meaningful is the difference overall? (edited as a response to @whuber) : This depends on how you use it. For instance, if the green CDF represented the CDF of some reference distribution and the red CDF was an empirical sample distribution, then the point by point vertical differences can be used in a Kolmogorov–Smirnov test for equality between the two distributions.

The fact that the green "leads" the red and the two curves are similarly shaped contribute to the fact that the green is always above the red, but this does not necessarily have to be the case. Consider that your populations do not come from the same underlying distribution. In this case the shape of the CDF would differ and the fact that the green "leads" the red would not necessarily result in the green always being above the red. For example, here are various CDFs of a logistic distribution (from Wikipedia)

CDF of logistic distributions

Notice that the red curve in the plot above "leads" (starts from a nonzero value) before the rest of the curves do, but ultimately end up below most of the curves as the x-values approach x=20.

  • 4
    $\begingroup$ The K-S test does not use areas: it uses maximum vertical discrepancies between a CDF and an empirical CDF. $\endgroup$
    – whuber
    Commented Aug 23, 2017 at 13:03
  • $\begingroup$ @whuber agreed. I suppose I should be more clear in my response. $\endgroup$
    – joceratops
    Commented Aug 23, 2017 at 13:07
  • 1
    $\begingroup$ See my comments on colours, specifically red and green, on the original question. $\endgroup$
    – Nick Cox
    Commented Aug 23, 2017 at 14:25
  • $\begingroup$ joceratops: Many thanks as well! The underlying data are actually more complex. In your analogy of boys and girls and their scores, you can think of the boys and girls as actually being of different age. Calculating the shift of "all boys" versus "all girls", as is currently done, ignores this information completely; is there a way of taking it into account? $\endgroup$ Commented Aug 24, 2017 at 5:56
  • $\begingroup$ In fact, a better analogy to the data I have is children of different age (that's the x-axis, older to younger from left to right)), and their height. Children older than 10 years, say, are green; the others are red. The shift confirms that older children are taller. Nevertheless, the data I have may be misleading: There may e.g. exist a weird, opposing trend within the teenagers, among which the rule could be "the younger the taller". $\endgroup$ Commented Aug 24, 2017 at 5:57

The absolute value of this area is $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ which note – at least for continuous distributions – is exactly equal to $$\int_{x=-\infty}^\infty \lvert F^{-1}(x) - G^{-1}(x)\rvert \,\mathrm{d}x.$$ In one dimension, the latter is the 1-Wasserstein distance, the 1-Kantorovich distance, or the "earth-mover's distance." It is quite a reasonable distance between probability distributions, which is easy to compute between one-dimensional distributions based on their cdfs.

For multivariate distributions, there is a natural extension (not based on CDFs, which become hard to work with), most commonly defined based on optimal transport. You can think of it this way: think of each density function as a pile of dirt. The amount of dirt that you need to move to transform one density into another is exactly this distance. This leads to the name of "earth-mover's distance."

It's not entirely obvious at first that this distance corresponds to the difference in area between CDFs. But imagine doing this for two point mass distributions, one at $x$ and one at $x'$; the area between their CDFs is a rectangle, with area $1 \times \lvert x - x' \rvert$, exactly the amount that you need to move the "dirt." You can then envision doing the same thing for a collection of point masses, getting a series of rectangles that you add up. When you go to continuous distributions in the limit, you get the integral written above, and it should hopefully make sense that these are the same thing.

The traditional way to estimate this distance from samples is by directly computing this transportation problem with a linear program, though there are more recent fast approximations.

The beautiful Kantorovich-Rubinstein duality also applies to this distance. This relation has this year has led to an explosion of interest in the Wasserstein distance among the deep learning community, via this paper which uses it for generative modeling. The distance has also been popular in computer vision applications for decades.

  • 4
    $\begingroup$ +1. For clarity, please note that your expression is not an "integral of this area": it simply is the (absolute) area. $\endgroup$
    – whuber
    Commented Aug 23, 2017 at 16:14
  • 1
    $\begingroup$ Dougal: Many thanks for your answer! Thinking in the framework of "moving dirt" is making things much clearer and intuitive. However,for this description to work, you need to visualize the data as histograms (piles of dirt), right? Or, is there also an intuitive analogy for the shaded area in my figure, with direct reference to the CDFs? (See also my followup below to joceratops). $\endgroup$ Commented Aug 24, 2017 at 6:00
  • $\begingroup$ I added a description of how to visualize this as a CDF area. Hopefully it makes sense; in f not, I can try to come back and make some pictures later. :) $\endgroup$
    – Danica
    Commented Aug 24, 2017 at 8:35
  • 2
    $\begingroup$ @Dougal Yes, I see this now. Intuitively, the first expression (absolute difference of CDFs) integrates "horizontally" by adding up vertical slices between $F(x)$ and $G(x)$, while the second expression integrates "vertically" by adding up horizontal slices between $F^{-1}(x)$ and $G^{-1}(x)$. $\endgroup$
    – Pearl
    Commented May 14, 2020 at 19:20
  • 1
    $\begingroup$ @Aubergine For continuous distributions they're the same thing; I don't think they are for discrete distributions, and so doing it on an empirical CDF versus an empirical inverse-CDF may give you different answers (not 100% sure without thinking about it more here). It's the inverse CDF that agrees with the Wasserstein distance. $\endgroup$
    – Danica
    Commented Sep 29, 2021 at 1:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.