0
$\begingroup$

I've noticed that when I sample n numbers from a continuous distribution D, sort them, and plot against the quantile function of D, both curves seem very similar. I've tried this for a few distributions (Gaussian, Weibull, Uniform, Gamma)) and I've seen the same behavior in all of them. Is there any connection between both?

$\endgroup$
  • 1
    $\begingroup$ For related threads search on "probability plot," "qq plot," and "probability integral transform." $\endgroup$ – whuber May 18 '18 at 11:57
1
$\begingroup$

The reason you are observing that behavior is because you are essentially creating two samples from the same distribution, just in a different way. When you simulate from the distribution, you are creating one sample, say $A$, which you would probably expect to show you all possible values in the range $\mathcal{R}(X)$ of the random variable $X$.

The inverse of the density function $F_X(x)$ is called the quantile function, let's denote it by $F_X^{-1}(p)$. So, where $F$ takes as input a realization $x$ of the rv $X$ and gives you the probability $\mathbb{P}[X\leq x]$, the quantile function does the inverse, i.e. it takes as input a probability level $p$ and gives you the minimum value of $x$ such that the probability of $X\leq x$ is at least $p$. Mathematically, this is formulated as $$F_X^{-1}(p) = \min\{x\in\mathcal{R}(X): p\leq F_X(x)\},\text{ }p\in[0,1]$$ (If $F_X$ is continuous, the inequality is substituted by an equality.)

This means that if you calculate the quantile function $\forall p\in[0,1]$, what you will get as a result is the full $\mathcal{R}(X)$, i.e. all possible values of $X$, which consitutes your "sample" $B$. Now, if your simulation for $A$ was good enough, namely $n$ was large enough to get a good idea about the possible values of $X$, then $A$ will be very close to $B$, so plotting them against each other would give you a line very close to the identity $y = x$. Of course, the more you increase your sample size $n$, the more "certain" you can be that you've seen all possible values of $X$, and the closer the plot will resemble this line.

For distributions with known densities, you can think of $A$ as an approximation to $B$, assuming that, as you say, you sort the results of $A$ and not count those truly unlikely instances where you have duplicates in $A$ :)

$\endgroup$
  • $\begingroup$ That's a great answer, thanks. Additionally, does that mean that if I have sorted samples that I suspect were generated by some distribution, I may have a hint to this distribution by looking at the plot of random samples, right? But I guess that wouldn't be very useful, since a few quantile functions from different distributions are quite similar, and plus, it is just easier to take the histogram after all :p $\endgroup$ – Alberto A May 18 '18 at 15:31
  • $\begingroup$ Not entirely sure what you mean in your first sentence, but in general there is little use in sorting the sample apart from a) calculating the ecdf (which is done by ready-made functions in statistical software anyway), or b) looking at problems involving ordered statistics. If your goal is to identify the distribution of your sample, you can look at qqplots, or tests like the Kolmogorov-Smirnov. $\endgroup$ – Emil May 19 '18 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.