"Intuitive" Understanding of the Probability Integral Transform [duplicate]

Question

I am trying to find a more "intuitive" understanding of the Probability Integral Transform (for the sake of better understanding Copula Models).

As far as I understand, the Probability Integral Transform is used for relating any continuous probability distribution to the uniform probability distribution. This transform states that the inverse of the cumulative probability distribution function of any probability distribution follows a uniform probability distribution.

For example, take the Normal Distribution (any probability distribution could have been chosen):

Then, take the Cumulative Normal Distribution:

The inverse of this Cumulative Distribution will follow a Uniform Probability Distribution:

Example:

I tried to show this using the R programming language.

Below is an Exponential Distribution converted into a Uniform Distribution:

x <- rexp(10000, 0.5)
y <- 1 - exp(-0.5*x)

hist(x)
hist(y)

And here is a Uniform Distribution converted into an Exponential Distribution:

x <- runif(10000)
y <- log(-x + 1) / (-1*0.5)

hist(x)
hist(y)

My Question: I have seen the mathematical proof of this Transform - but is there an "intuitive explanation" that explains why the Uniform Probability Distribution relates so many Probability Distributions together?

Thanks!

References:

• https://en.wikipedia.org/wiki/Probability_integral_transform
• https://en.wikipedia.org/wiki/Normal_distribution
• https://en.wikipedia.org/wiki/Continuous_uniform_distribution

Answer 1

First, one clarification: the probability integral transform uses the CDF, not the inverse CDF.

Here's how I like to think of it. Say we take a bunch of samples from a $N(0,1)$ distribution. Clearly these will be distributed $N(0,1)$ and we'll wind up with a bell-shaped histogram with enough draws.

Then say that $F$ is the cdf of a $N(0,1)$ random variable, and we take our samples and plug each of them into $F$. Now we have converted our original draws into their percentile values. For example, a draw of 0.0 = 50th percentile, a draw of 1.0 = 84th percentile, a draw of -1.96 = 2.5th percentile, etc.

So what's the probability that a draw from a $N(0,1)$ distribution will be in the 95th percentile or higher (of the underlying distribution, not the sample)? By definition, 5%. What's the probability it will be between the 50-60th percentiles? 10%. What's the probability it will be at or below the 1st percentile? 1%.

And this is true for any distribution, not just the normal distribution.

Answer 2

The correct statement is that, for any probability cumulative distribution $F$ (in dimension one), defining the generalised inverse as $$F^-(u)=\inf\{x;~F(x)\ge u\}$$ the random variable$$X=F^-(U)\sim F\tag{1}$$ meaning it is distributed from the distribution associated with $F$.

When $F$ is continuous and strictly increasing, this implies that $$F(X)\sim\mathcal U(0,1)\quad\text{when}\quad X\sim F\tag{2}$$ but this property does not hold for discrete and mixed variables $X$.

When (2) hold, the mathematical explanation is rather intuitive: $$\mathbb P(F(X)\le F(x)) = \mathbb P(X\le x) = F(x)$$