# Given that one can sample $X \sim f(x)$, is there an easy way to sample $Y \sim k \cdot f(g(y))$ (such as $k \cdot f(e^y)$)?

Say I'm able to sample an RV $X$ from a PDF $f(x)$, can I exploit this to efficiently sample another RV $Y \sim k \cdot f(g(y))$ (where $k$ is a normalizing constant)?

I'm interested in something for a Gibbs sampler that would be better than using Metropolis, slice, inverse transform sampling, etc.

• I don't think there should be a reasonable way to do this in general with a single draw from $f$. You either need some special conditions on $g(x)$ and $f(x)$ or be willing to settle for an approximation, considering that if you could get exact samples like this in general then you could devise samplers for getting a sample from any distribution by choosing $g$ and $f$ appropriately. I'm also not hopeful for your related problem since it would imply that we could simulate from $f(\theta | x)$ if $\theta$ is real provided $f(\theta)$ is easy to sample from and $f(x|\theta)$ is symmetric.
– guy
Apr 14, 2013 at 5:18
• Could you post the 'related question' as a different question? (and then put that link here) Apr 14, 2013 at 13:14
• Is $f \circ g$ similar to $f$? If so, this is easy. I guess not though. Can you tell us anything more? Apr 14, 2013 at 13:15
• @AaronMcDaid: Good idea, done. Apr 14, 2013 at 15:48

If the inverse of $g(y)$, i.e. $g^{-1}(x)$, is relatively linear for the most probable values drawn from $f(x)$, then the following method should have a reasonable acceptance rate. I'm assuming $g(y)$ is strictly monotonic - in order that it have an inverse.

This is Metropolis-Hastings. We make a proposal, and then accept or reject appropriately.

Make a draw from your sampler, $x \sim f(x)$, and feed that into the inverse to get a proposal $y_{proposal} = g^{-1}(x)$. The probability of making that proposal is proportional to $$\propto \frac{f(x)}{ \left| \frac{d}{dx} g^{-1}(x) \right| }$$

That expression uses the derivative of the inverse to account for the fact that it will overrepresent areas where g^{-1} is relatively constant.

We must calculate the proposal probabilty for both the proposed value, $y_{proposal}$, and the current value, $y_{current}$. We can also use the definitions $x_{proposal} = g(y_{proposal})$ and $x_{current} = g(y_{current})$.

$$Acceptance~probability = \operatorname{min} \left(1, \frac{ f(g(y_{proposal})) }{ f(g(y_{current})) } \frac{ \frac{f(x_{current})}{ \left| \frac{d}{dx} g^{-1}(x_{current}) \right| } }{ \frac{f(x_{proposal})}{ \left| \frac{d}{dx} g^{-1}(x_{proposal}) \right| } } \right)$$

Don't worry about the absolute value $|\cdot|$ in that equation. $g(y)$ is either increasing everywhere or decreasing everywhere and therefore the absolute values will cancel out.

We can do a lot of cancelling here, in particular remember that $x = g(y)$.

$$Acceptance~probability = \operatorname{min} \left(1, \frac{ f(x_{proposal}) }{ f(x_{current}) } \frac{ \frac{f(x_{current})}{ \left| \frac{d}{dx} g^{-1}(x_{current}) \right| } }{ \frac{f(x_{proposal})}{ \left| \frac{d}{dx} g^{-1}(x_{proposal}) \right| } } \right)$$

$$Acceptance~probability = \operatorname{min} \left(1, \frac{ { \frac{d}{dx} g^{-1}(x_{proposal}) } }{ { \frac{d}{dx} g^{-1}(x_{current}) } } \right)$$

Finally, I think you can rearrange a little more, and make use of the fact that the derivative of the inverse is the reciprocal of the derivative of the original function:

$$Acceptance~probability = \operatorname{min} \left(1, \frac{ { \frac{d}{dy} g(y_{current}) } }{ { \frac{d}{dy} g(y_{proposal}) } } \right)$$