Why is the CDF of a sample uniformly distributed

Question

I read here that given a sample $ X_1,X_2,...,X_n $ from a continuous distribution with cdf $ F_X $, the sample corresponding to $ U_i = F_X(X_i) $ follows a standard uniform distribution.

I have verified this using qualitative simulations in Python, and I was easily able to verify the relationship.

import matplotlib.pyplot as plt
import scipy.stats

xs = scipy.stats.norm.rvs(5, 2, 10000)

fig, axes = plt.subplots(1, 2, figsize=(9, 3))
axes[0].hist(xs, bins=50)
axes[0].set_title("Samples")
axes[1].hist(
    scipy.stats.norm.cdf(xs, 5, 2),
    bins=50
)
axes[1].set_title("CDF(samples)")

Resulting in the following plot:

I am unable to grasp why this happens. I assume it has to do with the definition of the CDF and it's relationship to the PDF, but I am missing something ...

I would appreciate it if someone could point me to some reading on the subject or help me get some intuition on the subject.

EDIT: The CDF looks like this:

Answer 1

Assume $F_X$ is continuous and increasing. Define $Z = F_X(X)$ and note that $Z$ takes values in $[0, 1]$. Then $$F_Z(x) = P(F_X(X) \leq x) = P(X \leq F_X^{-1}(x)) = F_X(F_X^{-1}(x)) = x.$$ The derivative of $F_Z$ is constant so $Z$ is uniformly distributed.

A more specific way to see this is by observing that for a uniform random variable $U$ taking values in $[0, 1]$, $F_U(x) = \int_R f_U(u)\,du =\int_0^x \,du =x$ so $F_Z(x) = F_U(x)$ for every $x\in[0, 1]$. Since $Z$ and $U$ has the same distribution function $Z$ must also be uniform on $[0, 1]$.

Answer 2

Intuitively, perhaps it makes sense to think of $F(x)$ as a percentile function, e.g. $F(x)$ of a randomly generated sample from the DF $F$ is expected to fall below $x$. Alternately $F^{-1}$ (think inverse images, not a proper inverse function per se) is a "quantile" function. That is, $x = F^{-1}(p)$ is the point $x$ behind which falls $p$ proportion of the sample. The functional composition is measurably commutative $F \circ F^{-1} =_\lambda F^{-1} \circ F$.

The uniform distribution is the only distribution having a quantile function equal to a percentile function: they are the identity function. So the image space is the same as the probability space. $F$ maps continuous random variables into a (0, 1) space with equal measure. Since for any two percentiles, $a < b$, we have $P(F^{-1}(a) < x < F^{-1}(b)) = P(a < F(X) < b) = b-a$

Answer 3

Here's some intuition. Let's use a discrete example.

Say after an exam the students' scores are $X = [10, 50, 60, 90]$. But you want the scores to be more even or uniform. $h(X) = [25, 50, 75, 100]$ looks better.

One way to achieve this is to find the percentiles of each student's score. Score $10$ is $25\%$, score $50$ is $50\%$, and so on. Note that the percentile is just the CDF. So the CDF of a sample is "uniform".

When $X$ is a random variable, the percentile of $X$ is "uniform" (e.g. the number of $X$'s in $0-25$ percentile should be the same as the number of $X$'s in $25-50$ percentile). Therefore the CDF of $X$ is uniformly distributed.

Answer 4

As I commented under several posts above, the key of a rigorous (and succinct) proof to the general continuous $F$ (that is, $F$ is not necessarily strictly increasing) is by introducing the quantile function: \begin{align*} \varphi(u) = \inf\{x: F(x) \geq u\}, \quad 0 < u < 1. \tag{1}\label{1} \end{align*} Note that this function is well defined for any distribution function $F$, regardless it has discontinuities or strictly increasing. This can also be used as a definition of the population $u$-quantile of a random variable whose distribution function is $F$. A graph of $\varphi(\cdot)$, which stresses its values at a discontinuity and a plateau point, is shown as follows (source: Probability and Measure (3rd edition), p.189):

The most important property of $\varphi$ is its relationship with $F$: for any $0 < u < 1$: \begin{align*} F(\varphi(u)-) \leq u \leq F(\varphi(u)), \tag{2}\label{2} \end{align*} where $F(\varphi(u)-) = \lim\limits_{x \uparrow \varphi(u)}F(x)$. From the graph, inequality $\eqref{2}$ is evident. The proof of $\eqref{2}$ will be relegated to the end of this answer. Note that for a general $F$, $\eqref{2}$ is not equivalent to $F(\varphi(u)) = u$, as many "proofs" falsely claimed or relied on.

However, under the condition that $F$ is continuous (everywhere), $\eqref{2}$ of course reduces to $F(\varphi(u)) \leq u \leq F(\varphi(u))$, whence $F(\varphi(u)) = F(\varphi(u)-) = u$. It then follows that \begin{align*} P[F(X) < u] &= 1 - P[F(X) \geq u] \\ &= 1 - P[X \geq \varphi(u)] = P[X < \varphi(u)] = F(\varphi(u)-) = u. \tag{3}\label{3} \end{align*} In the second equality above, we used the definition $\eqref{1}$. Now by $\eqref{3}$ and the continuity (from above) of the probability measure, we have for $0 < u < 1$: \begin{align*} P[F(X) \leq u] = \lim\limits_{h \downarrow 0}P[F(X) < u + h] = \lim\limits_{h \downarrow 0}(u + h) = u. \end{align*} This shows that $F(X) \sim U(0, 1)$, and the proof is complete.

---

Proof of $\eqref{2}$

By definition $\eqref{1}$, if $y < \varphi(u)$, then $F(y) < u$ (otherwise, $\varphi(u)$ would not be a lower bound of the set $S_u := \{x: F(x) \geq u\}$ as $F(y) \geq u$ and $y < \varphi(u)$). This implies that $F(\varphi(u)-) = \lim\limits_{y \uparrow \varphi(u)}F(y) \leq u$.

On the other hand, for every $n \in \mathbb{N}$, since $\varphi(u) + n^{-1}$ is not the infimum of $S_u$, there exists $x_n \in S_u$ such that $x_n < \varphi(u) + n^{-1}$, whence $F(\varphi(u) + n^{-1}) \geq F(x_n) \geq u$ by the monotonicity of $F$. This implies by the right-continuity of $F$ that $F(\varphi(u)) = \lim\limits_{n \to \infty}F(\varphi(u) + n^{-1}) \geq u$.

This completes the proof of $\eqref{2}$.

Answer 5

Yes, CDF of a random variable has uniform distribution in the interval $(0, 1)$ because the distribution of its CDF has a form $F(Y \leq y) = y.$ In a formal way, if $X $ is a random variable, then $ F(X) \sim \textrm{Unif}(0,1). $

Proof:

To understand this special property, the key is to understand the inverse function which is generated from the relationship between $ X $ and its CDF. For example, if $ X$ is a Pareto distribution with parameter a and has CDF as:

$$ F(x)= \begin{cases} 1 - 1/ x^a, &\textrm{if}~ x > 1,\\ 0 &\textrm{if}~ x \leq 1\end{cases}$$

The inverse function is $ y = F(x) = 1 - 1/x^a.$

Thus we can know $x$ if we know $y$ and vice versa, leading us to the inverse function:

$$ x = 1/(1-y)^{1/a}$$

in a formal way: $F^{-1}$ is the inverse function of $F.$ If $F(x) = y $ then $F^{-1}(y) = x.$ In the above example:

$$ F^{-1}(y) = 1/(1-y)^{1/a} $$

For an easier example, if we have half of people score below 70 then its CDF $F(70) = 1/2,$ its inverse function $F^{-1}(1/2) = 70.$ Now we can see the act of inversing a function is to change from x to y and from $y$ to $x.$ Thus $F^{-1}(F(x)) = x$ and $F^{-1}(F(y)) = y.$

Now we are able to generate the proof we need. X is random variable. We have $F(x) = y $then:

$F^{-1}(y) = x,$ and $F(F^{-1}(y)) = y $ because it is just another way to say $F(x) = y . $

We find the distribution of variable Y by finding its cumulative distribution:

$$\Pr(Y < y) = \Pr(F(X) < y) = \Pr(X < F^{-1}(y)) = F(F^{-1}(y)) = y;$$

therefore $\Pr(Y < y) = y $ which makes it a uniform distribution $\textrm{Unif}(0,1). $