Help me understand the quantile (inverse CDF) function

Question

I am reading about the quantile function, but it is not clear to me. Could you provide a more intuitive explanation than the one provided below?

Since the cdf $F$ is a monotonically increasing function, it has an inverse; let us denote this by $F^{−1}$. If $F$ is the cdf of $X$, then $F^{−1}(\alpha)$ is the value of $x_\alpha$ such that $P(X \le x_\alpha) = \alpha$; this is called the $\alpha$ quantile of $F$. The value $F^{−1}(0.5)$ is the median of the distribution, with half of the probability mass on the left, and half on the right. The values $F^{−1}(0.25)$ and $F^{−1}(0.75)$ are the lower and upper quartiles.

Answer 1

All this may sound complicated at first, but it is essentially about something very simple.

By cumulative distribution function we denote the function that returns probabilities of $X$ being smaller than or equal to some value $x$,

$$ \Pr(X \le x) = F(x).$$

This function takes as input $x$ and returns values from the $[0, 1]$ interval (probabilities)—let's denote them as $p$. The inverse of the cumulative distribution function (or quantile function) tells you what $x$ would make $F(x)$ return some value $p$,

$$ F^{-1}(p) = x.$$

This is illustrated in the diagram below which uses the normal cumulative distribution function (and its inverse) as an example.

Example

As an simple example, you can take a standard Gumbel distribution. Its cumulative distribution function is

$$ F(x) = e^{-e^{-x}} $$

and it can be easily inverted: recall natural logarithm function is an inverse of exponential function, so it is instantly obvious that quantile function for Gumbel distribution is

$$ F^{-1}(p) = -\ln(-\ln(p)) $$

As you can see, the quantile function, according to its alternative name, "inverts" the behaviour of cumulative distribution function.

Generalized inverse distribution function

Not every function has an inverse. That is why the quotation you refer to says "monotonically increasing function". Recall that from the definition of the function, it has to assign for each input value exactly one output. Cumulative distribution functions for continuous random variables satisfy this property since they are monotonically increasing. For discrete random variables cumulative distribution functions are not continuous and increasing, so we use generalized inverse distribution functions which need to be non-decreasing. More formally, the generalized inverse distribution function is defined as

$$ F^{-1}(p) = \inf \big\{x \in \mathbb{R}: F(x) \ge p \big\}. $$

The definition, translated to plain English, says that for given probability value $p$, we are looking for some $x$, that results in $F(x)$ returning value greater or equal then $p$, but since there could be multiple values of $x$ that meet this condition (e.g. $F(x) \ge 0$ is true for any $x$), so we take the smallest $x$ of those.

Functions with no inverses

In general, there are no inverses for functions that can return same value for different inputs, for example density functions (e.g., the standard normal density function is symmetric, so it returns the same values for $-2$ and $2$ etc.). The normal distribution is an interesting example for one more reason—it is one of the examples of cumulative distribution functions that do not have a closed-form inverse. Not every cumulative distribution function has to have a closed-form inverse! Hopefully in such cases the inverses can be found using numerical methods.

Use-case

The quantile function can be used for random generation as described in How does the inverse transform method work?

Answer 2

Tim had a very thorough answer. Good job!

I'd like to add one more remark. Not every monotonically increasing function has an inverse function. Actually only strictly monotonically increasing/decreasing functions have inverse functions.

For monotonically increasing cdf which are not strictly monotonically increasing, we have a quantile function which is also called the inverse cumulative distribution function. You can find more details here.

Both inverse functions (for those strictly increasing cdf) and quantile functions (for those monotonically increasing but not strictly monotonically increasing cdfs) can be denoted as $F^{-1}$, which can be confusing sometimes.

Answer 3

Chapter 2 of the book "Statistical Distributions" by Forbes, Evans, Hastings, and Peacock has a concise summary with consistent notation.

A quantile is any possible value (e.g. in context of a random draw) of a variable, that is, a variate. The authors give an example of a sample space of tossing 2 coins as the set {HH, HT, TH, TT}. The number of heads in that sample is a quantile of the ordered set {0, 1, 2}.

For a probability distribution or mass function, you are plotting the variate on the x-axis and the probability on the y-axis.

If you knew the probability and the function and wanted to deduce the variate on the x-axis from it, you would invert the function or approximate an inversion of it to get x, knowing y.

The discrete or continuous values along the y-axis for the discrete or continuous pdf might not be increasing and there may be multiple x's which would result in the same y.

The CDF (cumulative distribution function) is more convenient as the function plotted is increasing along the x-axis and the y-axis. Extracting the quantile, that is, the variate from CDF is usually easier math.

There are a few diagrams in the book demonstrating properties of the discrete probability distribution, and the CDF in chapter 2 and those are shown in answers posted to your question above this one also (though I can't see them while I'm typing this answer).

Table 2.1 has a concise summary of many terms and item 4 is for the inverse distribution function or quantile function (of probability alpha) and refers to determining x from the inverse function which takes the probability as an argument.

The book is a practical handbook on the subject with examples, though implementing the inverse functions requires other resources (like pre-computed tables findable at NIST or published approximation algorithms etc. https://www.itl.nist.gov/div898/handbook/eda/section3/eda367.htm).

(NOTE: everything past the 1st sentence was added in response to the comment from gung.)