# Sampling from the inverse of a distribution

Let's say I have some PDF $D$ that I can sample from. I would instead like to sample values from $Inv(D)$. I have that the $D$ is only positive within the bounds $[a,b]$ and is bounded above and below by $[c,d]$. That is, the domain of $D$ is a subset of $[a,b]$ and the range of $D$ is a subset of $[c,d]$. How can I go about sampling from $Inv(D)$?

I'm not sure that this is the standard use of "inverse" with respect to a distribution, so the image below should give an idea of what I mean. I can readily sample from the shaded region in the top distribution, but what I want is to sample from the shaded region in the bottom distribution. Update: I am looking to sample a single value from this inverse distribution at random intervals. My application is a setting wherein I have an agent exploring a domain. The agent models the areas of the value space that it has already explored, currently with a maximum-likelihood Gaussian of the values the agent has observed. Thus to pick a value of an unexplored area, the agent needs to periodically sample from the "inverse" of this Gaussian. Right now I'm just sampling from a uniform distribution and seeing decent results, so speed is more important to me than accuracy.

• There are myriad ways to do this, depending on how the PDF is given to you and whether you need a large sample. Could you edit this post to provide this information as well as to explain the purpose of this exercise? What kind of statistical problem is it intended to solve? – whuber Nov 10 '15 at 21:17
• @whuber I've updated the post per your suggestion. – Jake Nov 10 '15 at 22:07
• @Jake if there is some applied context to this problem you might do well to describe what that is, your comments on the other answer seem to indicate you have some "veiled" plan behind this... I can't even imagine what the applied context might be. – AdamO Nov 10 '15 at 22:28
• @AdamO See my update above. – Jake Nov 11 '15 at 2:05
• @Jake are you trying to implement an algorithm "checker" for an optimization routine that is likely to sample local maximae? So you want to problematically check the space for areas of lower likelihood to identify possible maximae that were overlooked? – AdamO Nov 11 '15 at 23:27

If you know the PDF (which I hope is the case) then you can use rejection sampling.

Assume your random variable has support on $[a, b]$ and has density $f (X)$, you can then sample $U_1 \sim U (a, b)$ and $U_2 \sim U (0,\max{(f(x))})$. If $U_2 > f (U_1)$ then accept $U_1$ as your sample from the "inverse" distribution. If not, start again.

• I have thought about it a bit more and I think the method is in principle good, but you need to draw U2 from a range [0, max(f(X))] – tristan Nov 11 '15 at 7:04
• If you are using a truncated Gaussian you can ignore the normalising factor when using rejection sampling, so f(x) doesn't have to be a proper PDF. – tristan Nov 11 '15 at 8:37

What you've depicted in your image is not an inverse distribution.

In fact, if we're to assume that what you're presenting can be functionally understood as $f_Y(x) = \mbox{Mode}(X) - f_X(x)$, then $f_Y$ can never be a proper density for a random variable unless you restrict it's support.

If the random variable has measurably finite support, and you use scaling the density of the $f_Y$ so that it integrates to 1, then this can be done.

The best way of doing so is simply coming up with a functional expression for this "shadow" distribution (hesitant to call it inverse), calculate its antiderivative, and sample from this functional using inverse transformations of uniform random variables.

• Thanks for your answer, and apologies for the terminology. I was hoping to be able to do this algorithmically without calculating an antiderivative. In any case, could you give an example for how to sample from the "shadow" of a Gaussian? – Jake Nov 10 '15 at 21:46
• @Jake No, a Gaussian distribution has support on the real line. – AdamO Nov 10 '15 at 21:47
• My mistake - I meant a truncated Gaussian normalized so that its PDF sums to 1 over the interval $[a,b]$. i.e. the CDF is 0 at $a$ and 1 at $b$. – Jake Nov 10 '15 at 21:51
• Assuming it's a symmetric centered interval, you'd use the inverse distribution rule to sample from the following CDF: $F(x) = \int_{a}^x \frac{\phi(0) - \phi(s)}{\Phi(b) - \Phi(a)}\mathcal{I}_{s \in [a, b]}ds$ – AdamO Nov 10 '15 at 21:58
• I'm not familiar with the notation you're using. What are $\phi$ and $\Phi$? Additionally, I'm not sure of how to go about finding $x$ even if I have a closed-form CDF. Would you use something like Newton's method? – Jake Nov 10 '15 at 22:05