6
$\begingroup$

For any random variable $X$ whose density is $\mathbb{P}(X=x)=p(x;\theta),$ where $\theta$ is a parameter, its deterministic function representation is $X=f(\theta, \omega)$ where $\omega$ is a random variable whose distribution is independent of $\theta$.

Question: what is the condition on $p(x; \theta)$ so that its deterministic function representation exists?

For example, if $X\sim N(\mu,\sigma^2)$, then $X=\mu+\sigma \omega$ where $\omega$ is a standard normal random variable.

If $X \sim \operatorname{Bernoulli}(q)$, then $X=1(\operatorname{logit}^{-1}(q)>\omega)$ where $\omega$ is a logistic random variable.

I don't know if there is a deterministic function representation for a Poisson random variable.

$\endgroup$
5
  • 1
    $\begingroup$ If $\omega$ is uniform$(0, 1)$ then I believe such a function $f$ always exists in accordance with the probability transform. en.wikipedia.org/wiki/Probability_integral_transform $\endgroup$
    – dsaxton
    Commented Dec 8, 2015 at 20:06
  • $\begingroup$ Yes but it may depend on the parameter $\theta$. The question is whether one can have a separation. $\endgroup$
    – hchen
    Commented Dec 8, 2015 at 20:09
  • 1
    $\begingroup$ In your post you required only that the distribution of $\omega$ not depend on $\theta$. Obviously $f$ should depend on $\theta$. $\endgroup$
    – dsaxton
    Commented Dec 8, 2015 at 20:11
  • $\begingroup$ Ah, I see your argument. Yes I believe this is true. $\endgroup$
    – hchen
    Commented Dec 8, 2015 at 20:16
  • $\begingroup$ It's a nice result because it suggests you can imagine there being a single primitive source of randomness, and all random variables as coming about by "stretching" or "twisting" that randomness in different ways. Also, applying it to the Bernoulli case gives the much simpler representation $X = I(U \leq q)$. $\endgroup$
    – dsaxton
    Commented Dec 8, 2015 at 20:24

1 Answer 1

1
$\begingroup$

This can be accomplished using inverse transformation sampling; it does not require a restriction on the desired distribution function. To do this, define $X \equiv f(\theta,\omega) \equiv \inf \{ r \in \mathbb{R} | F_\theta(r) \geqslant \omega \}$$^\dagger$ where $F_\theta$ is the desired distribution function conditional on the parameter $\theta$. Since the distribution $F_\theta$ is non-decreasing we have:

$$\inf \{ r \in \mathbb{R} | F_\theta(r) \geqslant \omega \} \leqslant x \quad \quad \iff \quad \quad F_\theta(x) \geqslant \omega .$$

Hence, taking $\omega \sim \text{U}(0,1)$ independent of the parameter $\theta$ you then get:

$$\begin{equation} \begin{aligned} \mathbb{P}(X \leqslant x | \theta) &= \mathbb{P} \Big( \inf \{ x \in \mathbb{R} | F_\theta(x) \geqslant \omega \} \leqslant x \Big| \theta \Big) \\[6pt] &= \mathbb{P} \Big( \omega \leqslant F_\theta(x) \Big| \theta \Big) \\[6pt] &= F_\theta(x), \\[6pt] \end{aligned} \end{equation}$$

which is the desired distribution for $X$ conditional on the parameter $\theta$. As you can see, there is no requirement on $F_\theta$ for this technique to work, though it is notable that it is not necessarily the most computationally efficient technique to generate your desired random variable. This technique can be extended to multivariate problems as shown in this related question.


$^\dagger$ This infimum function is the "generalised inverse" of $F_\theta$. In the case where $F_\theta$ is continuous you get $\inf \{ r \in \mathbb{R} | F_\theta(r) \geqslant \omega \} = F_\theta^{-1}(\omega)$, which is the inverse in the regular functional sense.

$\endgroup$
2
  • 1
    $\begingroup$ +1 I use an explicit example of this construction in an answer at stats.stackexchange.com/questions/352803/…. That example also shows how to extend this construction to multivariate distributions. $\endgroup$
    – whuber
    Commented Sep 17, 2018 at 13:58
  • $\begingroup$ @whuber: Thanks - I have edited to add a reference to that related question and answer. $\endgroup$
    – Ben
    Commented Sep 17, 2018 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.