6
$\begingroup$

For any random variable $X$ whose density is $\mathbb{P}(X=x)=p(x;\theta),$ where $\theta$ is a parameter, its deterministic function representation is $X=f(\theta, \omega)$ where $\omega$ is a random variable whose distribution is independent of $\theta$.

Question: what is the condition on $p(x; \theta)$ so that its deterministic function representation exists?

For example, if $X\sim N(\mu,\sigma^2)$, then $X=\mu+\sigma \omega$ where $\omega$ is a standard normal random variable.

If $X \sim \operatorname{Bernoulli}(q)$, then $X=1(\operatorname{logit}^{-1}(q)>\omega)$ where $\omega$ is a logistic random variable.

I don't know if there is a deterministic function representation for a Poisson random variable.

$\endgroup$
  • 1
    $\begingroup$ If $\omega$ is uniform$(0, 1)$ then I believe such a function $f$ always exists in accordance with the probability transform. en.wikipedia.org/wiki/Probability_integral_transform $\endgroup$ – dsaxton Dec 8 '15 at 20:06
  • $\begingroup$ Yes but it may depend on the parameter $\theta$. The question is whether one can have a separation. $\endgroup$ – hchen Dec 8 '15 at 20:09
  • 1
    $\begingroup$ In your post you required only that the distribution of $\omega$ not depend on $\theta$. Obviously $f$ should depend on $\theta$. $\endgroup$ – dsaxton Dec 8 '15 at 20:11
  • $\begingroup$ Ah, I see your argument. Yes I believe this is true. $\endgroup$ – hchen Dec 8 '15 at 20:16
  • $\begingroup$ It's a nice result because it suggests you can imagine there being a single primitive source of randomness, and all random variables as coming about by "stretching" or "twisting" that randomness in different ways. Also, applying it to the Bernoulli case gives the much simpler representation $X = I(U \leq q)$. $\endgroup$ – dsaxton Dec 8 '15 at 20:24
1
$\begingroup$

This can be accomplished using inverse transformation sampling; it does not require a restriction on the desired distribution function. To do this, define $X \equiv f(\theta,\omega) \equiv \inf \{ r \in \mathbb{R} | F_\theta(r) \geqslant \omega \}$$^\dagger$ where $F_\theta$ is the desired distribution function conditional on the parameter $\theta$. Since the distribution $F_\theta$ is non-decreasing we have:

$$\inf \{ r \in \mathbb{R} | F_\theta(r) \geqslant \omega \} \leqslant x \quad \quad \iff \quad \quad F_\theta(x) \geqslant \omega .$$

Hence, taking $\omega \sim \text{U}(0,1)$ independent of the parameter $\theta$ you then get:

$$\begin{equation} \begin{aligned} \mathbb{P}(X \leqslant x | \theta) &= \mathbb{P} \Big( \inf \{ x \in \mathbb{R} | F_\theta(x) \geqslant \omega \} \leqslant x \Big| \theta \Big) \\[6pt] &= \mathbb{P} \Big( \omega \leqslant F_\theta(x) \Big| \theta \Big) \\[6pt] &= F_\theta(x), \\[6pt] \end{aligned} \end{equation}$$

which is the desired distribution for $X$ conditional on the parameter $\theta$. As you can see, there is no requirement on $F_\theta$ for this technique to work, though it is notable that it is not necessarily the most computationally efficient technique to generate your desired random variable. This technique can be extended to multivariate problems as shown in this related question.


$^\dagger$ This infimum function is the "generalised inverse" of $F_\theta$. In the case where $F_\theta$ is continuous you get $\inf \{ r \in \mathbb{R} | F_\theta(r) \geqslant \omega \} = F_\theta^{-1}(\omega)$, which is the inverse in the regular functional sense.

$\endgroup$
  • 1
    $\begingroup$ +1 I use an explicit example of this construction in an answer at stats.stackexchange.com/questions/352803/…. That example also shows how to extend this construction to multivariate distributions. $\endgroup$ – whuber Sep 17 '18 at 13:58
  • $\begingroup$ @whuber: Thanks - I have edited to add a reference to that related question and answer. $\endgroup$ – Reinstate Monica Sep 17 '18 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.