Is there any relationship between random numbers sampled from some continuous distribution D and the quantile function of D?

Question

I've noticed that when I sample n numbers from a continuous distribution D, sort them, and plot against the quantile function of D, both curves seem very similar. I've tried this for a few distributions (Gaussian, Weibull, Uniform, Gamma)) and I've seen the same behavior in all of them. Is there any connection between both?

Answer 1

The reason you are observing that behavior is because you are essentially creating two samples from the same distribution, just in a different way. When you simulate from the distribution, you are creating one sample, say $A$, which you would probably expect to show you all possible values in the range $\mathcal{R}(X)$ of the random variable $X$.

The inverse of the density function $F_X(x)$ is called the quantile function, let's denote it by $F_X^{-1}(p)$. So, where $F$ takes as input a realization $x$ of the rv $X$ and gives you the probability $\mathbb{P}[X\leq x]$, the quantile function does the inverse, i.e. it takes as input a probability level $p$ and gives you the minimum value of $x$ such that the probability of $X\leq x$ is at least $p$. Mathematically, this is formulated as $$F_X^{-1}(p) = \min\{x\in\mathcal{R}(X): p\leq F_X(x)\},\text{ }p\in[0,1]$$ (If $F_X$ is continuous, the inequality is substituted by an equality.)

This means that if you calculate the quantile function $\forall p\in[0,1]$, what you will get as a result is the full $\mathcal{R}(X)$, i.e. all possible values of $X$, which consitutes your "sample" $B$. Now, if your simulation for $A$ was good enough, namely $n$ was large enough to get a good idea about the possible values of $X$, then $A$ will be very close to $B$, so plotting them against each other would give you a line very close to the identity $y = x$. Of course, the more you increase your sample size $n$, the more "certain" you can be that you've seen all possible values of $X$, and the closer the plot will resemble this line.

For distributions with known densities, you can think of $A$ as an approximation to $B$, assuming that, as you say, you sort the results of $A$ and not count those truly unlikely instances where you have duplicates in $A$ :)