2
$\begingroup$

I recently noticed that if you generate 10000 normally distributed numbers and then find the probability associated with each number (pnorm), each probability from 0 to 1 occurs with approximately the same frequency. Here's how I did it in R:

var2 <- numeric(10000)
normnos <- rnorm(10000)
for (i in 1:10000) {
  var2[i] <- pnorm(normnos[i])
}
hist(var2)

enter image description here

How is this possible? If all probabilities have equal likelihood of occurring, then wouldn't the resulting distribution be uniform instead of normal? I'm really confused and would appreciate an explanation.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
5
$\begingroup$
  1. pnorm doesn't compute the probability of the sampled number - it rather computes $P(X \leq x)$ - which is the cumulative distribution function. To compute the probability of sampled number, you will have to use the PDF - normal distribution in this case, that is, $p(x_i - \delta < X < x_i + \delta) = N(x_i | \mu = 0, \sigma = 1)$ ($\delta$ very small).
  2. The histogram you plotted is the distribution of cdf values, which is always uniform regardless of the distribution. This is known as "Universality of the Uniform"
  3. Mathematically, suppose $X$ is a random variable with pdf $p_X(x)$ and cdf $F_X(x) = P(X \leq x)$. Let $T$ be the random variable $T = F_X(X)$ - the samples of which you plotted in the histogram. $T$ is random because $X$ (normal variable in your case) is random. Then, $$F_T(t) = P(T \leq t) = P(F_X(X) \leq t) = P(X \leq F_X^{-1}(t)) = F_X(F_X^{-1}(t)) = t$$
  4. $F_T(t) = t$ -- this is the cdf of a uniform distribution. So, the pdf of T is uniform - which is what you plotted. Note that the inverse of $F_{X}(x)$ exists only if $F_X$ is continuous and strictly increasing.

Hope this helps! :)

Improve this answer
$\endgroup$
5
  • $\begingroup$ I see, it makes sense now. Thanks so much! :) $\endgroup$
    – Ryan Ma
    Commented Jan 3, 2021 at 8:04
  • 1
    $\begingroup$ Correction: The pdf at $x$, $\varphi(x)$, is not the probability of the event $X=x$, which is of probability zero. $\endgroup$
    – Xi'an
    Commented Jan 3, 2021 at 11:25
  • 1
    $\begingroup$ Correction #2: the cdf transform of a sample is only uniform when the cdf is continuous and strictly increasing. $\endgroup$
    – Xi'an
    Commented Jan 3, 2021 at 11:27
  • $\begingroup$ @Xi'an thanks for the corrections. the inverse of cdf can be taken only if it is continuous and strictly increasing. And for a continuous pdf, the probability of any single point is zero. Edited the answer. $\endgroup$
    – stbv
    Commented Jan 3, 2021 at 11:47
  • $\begingroup$ Not exactly: when using a generalised inverse, $F_X^{-}($ the result that $F_X^{-}(U)$ is distributed as $F_X$ is a universal result, as opposed to $F_X(X)$ being uniform only for an invertible $F_X$. $\endgroup$
    – Xi'an
    Commented Jan 3, 2021 at 13:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.