2
$\begingroup$

What is the percent point function (ppf), or inverse cdf, of the truncated normal distribution? The distribution and cdf is defined here: https://en.wikipedia.org/wiki/Truncated_normal_distribution $$F(x; \mu, \sigma,a, b) = \frac{\Phi(\frac{x-\mu}{\sigma})-\Phi(\frac{a-\mu}{\sigma})}{\Phi(\frac{b-\mu}{\sigma})-\Phi(\frac{a-\mu}{\sigma})}$$ $$F^{-1}(x; \mu, \sigma,\alpha, \beta) = ...?$$

My motivation is that I would like it implemented in Tensorflow Probability: https://github.com/tensorflow/probability/blob/master/tensorflow_probability/python/distributions/truncated_normal.py

$\endgroup$
6
  • 2
    $\begingroup$ The answer is also in the Wikipedia page. $\endgroup$
    – Xi'an
    Oct 7, 2020 at 15:55
  • $\begingroup$ @Xi'an, the question you post does not seem to touch on the ppf. $\endgroup$ Oct 7, 2020 at 16:44
  • 3
    $\begingroup$ The Wikipedia article you link to answers your question. $\endgroup$
    – whuber
    Oct 7, 2020 at 18:44
  • $\begingroup$ @whuber not as explicitly as I would like. I am noting the equation there, but I doesn't seem to define the ppf, does it? I get that there is some connection there, but I am not able to confidently extract the information I need, hence why I am asking here. Keep in mind my background is not from statistics. I am basically asking how you would complete the following equation: F^-1(x; μ, σ, α, b) = ... You'd do me a solid if you'd help me do this. $\endgroup$ Oct 8, 2020 at 8:04
  • $\begingroup$ What is your application for the truncated density? $\endgroup$
    – develarist
    Oct 8, 2020 at 8:18

3 Answers 3

5
$\begingroup$

As noted in comments, Wikipedia gives $$x = \Phi^{-1}( \Phi(\alpha) + U\cdot(\Phi(\beta)-\Phi(\alpha)))\sigma + \mu$$ for generating a random variate $x$ from a truncated normal distribution

Translating this expression to your question, I suspect you want $$F^{-1}(x; \mu, \sigma,\alpha,\beta) = \Phi^{-1}\left( \Phi(\alpha) + x\cdot(\Phi(\beta)-\Phi(\alpha))\right)\sigma + \mu$$ or $$F^{-1}(x; \mu, \sigma,a,b) = \Phi^{-1}\left( \Phi(\tfrac{a-\mu}{\sigma}) + x\cdot(\Phi(\tfrac{b-\mu}{\sigma})-\Phi(\tfrac{a-\mu}{\sigma}))\right)\sigma + \mu$$

$\endgroup$
2
  • $\begingroup$ Just came to the conclusion myself, after deriving it. Thank you so much, though! $\endgroup$ Oct 8, 2020 at 8:51
  • $\begingroup$ Your answer was significantly more beautifully typeset, though 🤩 $\endgroup$ Oct 8, 2020 at 8:53
5
$\begingroup$

We may draw insight from considering a generalization of truncation.

As a point of departure, then, I propose viewing truncation as an extreme example of locally modifying the probability. (This is known as change of measure in the literature on stochastic processes.)

That is, in the Wikipedia setting where a distribution $\lambda$ is truncated to the left of a number $a$ and the right of a number $b$ (either of which may be infinite, with an obvious meaning), all the probability of $\lambda$ is removed from the intervals $(-\infty,a)$ and $(b,\infty),$ leaving the rest of the probability unchanged in the set $\mathcal{A}=[a,b].$ We have to adjust that probability to make it total $1$ again. By multiplying it by a suitable constant (namely, the reciprocal of $\lambda(\mathcal A)$), we assure that the relative probabilities don't change.

(Because all this is best explained in terms of distributions rather than random variables, I have adopted a notation where the distribution is a set function: it assigns a probability to any measurable set $\mathcal A$. When $\lambda$ is the distribution of a random variable $X,$ this means $\lambda(\mathcal A) = \Pr(X\in\mathcal{A}).$)

A grand generalization of this creates a new distribution $\mu$ by systematically adjusting the original distribution $\lambda.$ This adjustment will be expressed as a function $f:\mathbb{R}\to\mathbb{R}^{+}$ that represents the factor to scale the probability. In the Wikipedia example, $f$ will be $0$ on the set $(-\infty, a)\cup (b,\infty),$ which is the complement $\bar{\mathcal{A}},$ and otherwise $f$ must be constant on $\mathcal{A}=[a,b]$ itself, where it equals $1/\lambda(\mathcal A).$ This removes all probability from the tails $(-\infty,a)$ and $(b,\infty).$

We may write

$$\mu = f\lambda.$$

The function $f$ is known as the Radon Nikodym derivative of $\mu$ with respect to $\lambda.$

Here is how we can exploit the derivative. The distribution function of a distribution $\lambda$ is, as usual, defined by

$$F_\lambda(x) = \lambda((-\infty, x]).$$

Given $F_\lambda$ and $f,$ you want to find an inverse distribution (aka quantile) function $F_{f\lambda}^{-1}.$ That is, given $q\in[0,1],$ we seek a number $y$ for which

$$(f\lambda)((-\infty, y]) = q.$$

The idea is to modify $q$ to another value $q^\prime$ for which $F_\lambda^{-1}(q^\prime)$ is guaranteed to lie in $\mathcal A$ with the desired relative probability.

The solution is so abstract it's trivial. Consider the curve $\gamma:\mathbb{R}\to[0,1]\times[0,1]$ given by

$$\gamma_{f,\lambda}(x) = (F_\lambda(x), F_{f\lambda}(x)).$$

As $x$ increases, this curve traces out the graph of a monotonic, non-decreasing function $\Gamma_{f,\lambda}:[0,1]\to[0,1]$ that connects $(0,0)$ to $(1,1)$ and

$$F_{f\lambda} = \Gamma_{f,\lambda} \circ F_\lambda$$

by construction. Moreover, wherever this curve is differentiable, its slope is the rate at which we are changing the probability; that is, for any $q^\prime \in [0,1]$ where $\Gamma_{f,\lambda}$ is differentiable,

$$\Gamma_{f,\lambda}^\prime(q^\prime) = f(F_\lambda^{-1}(q^\prime)).\tag{*}$$

Given any $q\in[0,1],$ then, set $q^\prime = \Gamma_{f,\lambda}^{-1}(q)$ (where $\Gamma_{f,\lambda}^{-1}$ is any functional right inverse of $\Gamma_{f,\lambda}$).

The value $x=F_\lambda^{-1}(q^\prime)$ is the desired $q$ quantile of $f\lambda:$

$$F_{f\lambda}^{-1} = F_\lambda^{-1} \circ \Gamma_{f,\lambda}^{-1}.$$

This is the percent point function of the modified distribution.

Let's make this concrete.

In the Wikipedia setting we can easily work out the function $\Gamma.$ Starting at extremely negative $x,$ initially the first coordinate of $\Gamma_{f\lambda}$ increases at the same rate the probability $F_\lambda(x)=\lambda((-\infty,x])$ increases, while (because there is no probability in the truncated distribution $f\lambda$ in this region) the second coordinate stays put at $0.$ At the point $x=a,$ both coordinates increase in tandem at a relative rate of $f(x) = 1 / \lambda([a,b]),$ according to equation $(*).$ Thus, $\Gamma_{f,\lambda}$ traces out a portion of a line as $x$ ranges from $a$ to $b.$ At $x=b,$ this curve again becomes horizontal in order to extend to $(1,1).$

When $\lambda$ is a continuous distribution, $\Gamma_{f,\lambda}$ is a continuous curve and looks like this:

Figure

This picture makes it clear that in the Wikipedia setting, an inverse of $\Gamma_{f,\lambda}$ is a linear function with slope $F_\lambda(b) - F_\lambda(a):$

$$\Gamma_{f,\lambda}^{-1}(q) = F_\lambda(a) + q\left(F_\lambda(b) - F_\lambda(a)\right).$$

As a test, let's truncate a standard Normal distribution at $a=-1/2$ and $b=1$ and draw a large number of independent samples from it using this solution.

Gamma.inverse <- function(q, a, b) {pnorm(a) + q * (pnorm(b) - pnorm(a))}
q <- runif(1e7)                  # Generate uniform percents `q`
q. <- Gamma.inverse(q, -1/2, 1)  # Convert to percents for the original distribution
x <- qnorm(q.)                   # Invert the original distribution

The simplicity of this solution attests to the underlying simplicity of the foregoing analysis.

Here is a histogram of the results on which I have plotted the truncated density (in red) and, for reference, a comparably scaled version of the original Normal density (in gray).

Figure 2

$\endgroup$
2
  • $\begingroup$ So, if I get this right: you have generalized the ppf for any truncated distribution? $\endgroup$ Oct 9, 2020 at 6:22
  • 1
    $\begingroup$ Harald, that is correct. I have gone far beyond that to include truncation on any set (not just one or two tails) and, in fact, arbitrary modifications of any distribution (so that if you can simulate data for one distribution, you can convert them into a simulation for another distribution that is absolutely continuous with respect to the first). This isn't magic, though: the process is effective only when it's easy to compute $\Gamma^{-1},$ for otherwise it's just an abstract and not terribly useful result. $\endgroup$
    – whuber
    Oct 9, 2020 at 13:25
2
$\begingroup$

$$F^{-1}(p; \mu, \sigma,a,b) = \Phi^{-1}\left(\Phi\left(\frac{a-\mu}{\sigma}\right) + p\left(\Phi\left(\frac{b-\mu}{\sigma}\right) - \Phi\left(\frac{a-\mu}{\sigma}\right)\right)\right)\sigma + \mu,\quad p \in (0, 1)$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.