8
$\begingroup$

I know that the proof of the probability integral transform has been given multiple times on this site. However, the proofs I found use the hypothesis that the CDF $F_X(x)$ is strictly increasing (together, of course, with the hypothesis that $X$ is a continuous random variable). I know that actually the only required hypothesis is that $X$ is a continuous random variable, and strict monotonicity is not required. Can you show me how?

Since I'm already here, I also take the occasion to ask for a simple application of the probability integral transform :) can you show me that, if $X$ has CDF $F_X(x)$ and $Y$ is the truncation of $X$ to $[a,b]$, then $Y$ is distributed as $F_X^{-1}(U)$ where $U\sim[F_X(a),F_X(b)]$?

$\endgroup$
  • 2
    $\begingroup$ if you would be so kind, in the proof of your link, could you point to where the requirement that $F_X(x)$ has to be strictly increasing. Thanks! $\endgroup$ – Erosennin Apr 29 '16 at 12:57
  • 1
    $\begingroup$ @Erosennin, the proof assumes the existence of the inverse of $F_X(x)$. $\endgroup$ – DeltaIV Apr 29 '16 at 13:23
  • $\begingroup$ Thanks! But is there ever a CDF that is not strictly increasing? You have probably already thought of this, though... $\endgroup$ – Erosennin Apr 29 '16 at 13:28
  • 2
    $\begingroup$ Of course there is. The random variable whose pdf is equal to 1/2 in [0,0.5], 0 in [0.5,1] and 1/2 in [1,1.5], has a CDF which is continuous, but is not strictly increasing. $\endgroup$ – DeltaIV Apr 29 '16 at 13:40
  • $\begingroup$ The hard part is dealing with the non-absolutely continuous part of $F$. The idea is made clear by considering the extreme case of discrete $F$. At stats.stackexchange.com/a/36246/919 I give an algorithm that implements the probability integral transform in that case (as well as supplying working code). Emulating that algorithm for arbitrary $F$ will answer your question. $\endgroup$ – whuber Apr 29 '16 at 14:38
9
$\begingroup$

In the wikipedia link provided by the OP, the probability integral transform in the univariate case is given as follows

Suppose that a random variable $X$ has a continuous distribution for which the cumulative distribution function(CDF) is $F_X$. Then the random variable $Y=F_X(X)$ has a uniform distribution.
PROOF
Given any random variable $X$, define $Y = F_X (X)$. Then:

$$ \begin{align} F_Y (y) &= \operatorname{Prob}(Y\leq y) \\ &= \operatorname{Prob}(F_X (X)\leq y) \\ &= \operatorname{Prob}(X\leq F^{-1}_X (y)) \\ &= F_X (F^{-1}_X (y)) \\ &= y \end{align} $$

$F_Y$ is just the CDF of a $\mathrm{Uniform}(0,1)$ random variable. Thus, $Y$ has a uniform distribution on the interval $[0, 1]$.

The problem with the above is that it is not made clear what the symbol $F_X^{-1}$ represents. If it represented the "usual" inverse (that exists only for bijections), then the above proof would hold only for continuous and strictly increasing CDFs. But this is not the case, since for any CDF we work with the quantile function (which is essentially a generalized inverse),

$$F_Z^{-1}(t) \equiv \inf \{z : F_Z(z) \geq t \}, \;\;t\in (0,1)$$

Under this definition the wikipedia series of equalities continue to hold, for continuous CDFs. The critical equality is

$$\operatorname{Prob}(X\leq F^{-1}_{X} (y)) = \operatorname{Prob}(X\leq \inf \{x : F_X(x) \geq y \})= \operatorname{Prob}(F_X (X)\leq y)$$

which holds because we are examining a continuous CDF. This in practice means that its graph is continuous (and without vertical parts, since it is a function and not a correspondence). In turn, these imply that the infimum (the value of inf{...}), denote it $x(y)$, will always be such that $F_X(x(y)) = y$. The rest is immediate.

Regarding CDFs of discrete (or mixed) distributions, it is not (cannot be) true that $Y=F_X(X)$ follows a uniform $U(0,1)$, but it is still true that the random variable $Z=F_{X}^{-1}(U)$ has distribution function $F_X$ (so the inverse transform sampling can still be used). A proof can be found in Shorack, G. R. (2000). Probability for statisticians. ch.7.

$\endgroup$
  • 2
    $\begingroup$ +1 A similar proof is also provided on pg. 54 of Casella and Berger's Statistical Inference, second edition. $\endgroup$ – StatsStudent Apr 30 '16 at 3:29
  • 1
    $\begingroup$ @Analyst1 Thanks, it's good to have multiple references. $\endgroup$ – Alecos Papadopoulos Apr 30 '16 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.