17
$\begingroup$

I read here that given a sample $ X_1,X_2,...,X_n $ from a continuous distribution with cdf $ F_X $, the sample corresponding to $ U_i = F_X(X_i) $ follows a standard uniform distribution.

I have verified this using qualitative simulations in Python, and I was easily able to verify the relationship.

import matplotlib.pyplot as plt
import scipy.stats

xs = scipy.stats.norm.rvs(5, 2, 10000)

fig, axes = plt.subplots(1, 2, figsize=(9, 3))
axes[0].hist(xs, bins=50)
axes[0].set_title("Samples")
axes[1].hist(
    scipy.stats.norm.cdf(xs, 5, 2),
    bins=50
)
axes[1].set_title("CDF(samples)")

Resulting in the following plot:

Plot showing the sample of a normal distribution and the cdf of the sample.

I am unable to grasp why this happens. I assume it has to do with the definition of the CDF and it's relationship to the PDF, but I am missing something...

I would appreciate it if someone could point me to some reading on the subject or help me get some intuition on the subject.

EDIT: The CDF looks like this:

CDF of the sampled distribution

$\endgroup$
  • 2
    $\begingroup$ Compute the cdf of $F_X(X)$. $\endgroup$ – Zhanxiong Jul 15 '15 at 16:50
  • 2
    $\begingroup$ You would find a proof of this property (for continuous rv's) in any book about simulation as this is the basis of the inverse cdf simulation method. $\endgroup$ – Xi'an Jul 15 '15 at 17:06
  • 2
    $\begingroup$ Also try google-ing probability integral transform $\endgroup$ – Zachary Blumenfeld Jul 15 '15 at 17:13
  • 1
    $\begingroup$ @Xi'an It is good to point out the conclusion holds only for continuous random variables. Sometimes this result is mistakenly used for discrete random variables. On the other hand, also note many proofs involves the step $P(F(X) \leq x) = P(X \leq F^{-1}(x))$ in which assumes the strict monotonicity of $F$, which is also a too strong assumption. The following link provides a rigorous summary on this topic:people.math.ethz.ch/~embrecht/ftp/generalized_inverse.pdf $\endgroup$ – Zhanxiong Jul 15 '15 at 17:20
  • $\begingroup$ @Zhanxiong the only condition necessary for $F$ is that it is càdlàg. $\endgroup$ – AdamO Jul 15 '15 at 17:26
18
$\begingroup$

Assume $F_X$ is continuous and increasing. Define $Z = F_X(X)$ and note that $Z$ takes values in $[0, 1]$. Then $$F_Z(x) = P(F_X(X) \leq x) = P(X \leq F_X^{-1}(x)) = F_X(F_X^{-1}(x)) = x.$$

On the other hand, if $U$ is a uniform random variable that takes values in $[0, 1]$, $$F_U(x) = \int_R f_U(u)\,du =\int_0^x \,du =x.$$

Thus $F_Z(x) = F_U(x)$ for every $x\in[0, 1]$.

$\endgroup$
  • $\begingroup$ Does it follow that Z has a uniform(0, 1) distribution? $\endgroup$ – StatsSorceress Nov 13 at 3:29
8
$\begingroup$

Intuitively, perhaps it makes sense to think of $F(x)$ as a percentile function, e.g. $F(x)$ of a randomly generated sample from the DF $F$ is expected to fall below $x$. Alternately $F^{-1}$ (think inverse images, not a proper inverse function per se) is a "quantile" function. That is, $x = F^{-1}(p)$ is the point $x$ behind which falls $p$ proportion of the sample. The functional composition is measurably commutative $F \circ F^{-1} =_\lambda F^{-1} \circ F$.

The uniform distribution is the only distribution having a quantile function equal to a percentile function: they are the identity function. So the image space is the same as the probability space. $F$ maps continuous random variables into a (0, 1) space with equal measure. Since for any two percentiles, $a < b$, we have $P(F^{-1}(a) < x < F^{-1}(b)) = P(a < F(X) < b) = b-a$

$\endgroup$
  • $\begingroup$ I struggled for hours, but finally it clicked why the derived random variable $Y = F(X)$ is uniformly distributed. Your answer really helped, thanks a lot. It seems very much like in algebra where 1 was the multiplicative identity. $\endgroup$ – Aditya P Sep 21 '18 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.