Given that one can sample $X \sim f(x)$, is there an easy way to sample $Y \sim k \cdot f(g(y))$ (such as $k \cdot f(e^y)$)?

Question

Say I'm able to sample an RV $X$ from a PDF $f(x)$, can I exploit this to efficiently sample another RV $Y \sim k \cdot f(g(y))$ (where $k$ is a normalizing constant)?

I'm interested in something for a Gibbs sampler that would be better than using Metropolis, slice, inverse transform sampling, etc.

Answer 1

If the inverse of $g(y)$, i.e. $g^{-1}(x)$, is relatively linear for the most probable values drawn from $f(x)$, then the following method should have a reasonable acceptance rate. I'm assuming $g(y)$ is strictly monotonic - in order that it have an inverse.

This is Metropolis-Hastings. We make a proposal, and then accept or reject appropriately.

Make a draw from your sampler, $x \sim f(x)$, and feed that into the inverse to get a proposal $y_{proposal} = g^{-1}(x)$. The probability of making that proposal is proportional to $$\propto \frac{f(x)}{ \left| \frac{d}{dx} g^{-1}(x) \right| } $$

That expression uses the derivative of the inverse to account for the fact that it will overrepresent areas where g^{-1} is relatively constant.

We must calculate the proposal probabilty for both the proposed value, $y_{proposal}$, and the current value, $y_{current}$. We can also use the definitions $x_{proposal} = g(y_{proposal})$ and $x_{current} = g(y_{current})$.

$$ Acceptance~probability = \operatorname{min} \left(1, \frac{ f(g(y_{proposal})) }{ f(g(y_{current})) } \frac{ \frac{f(x_{current})}{ \left| \frac{d}{dx} g^{-1}(x_{current}) \right| } }{ \frac{f(x_{proposal})}{ \left| \frac{d}{dx} g^{-1}(x_{proposal}) \right| } } \right) $$

Don't worry about the absolute value $|\cdot|$ in that equation. $g(y)$ is either increasing everywhere or decreasing everywhere and therefore the absolute values will cancel out.

We can do a lot of cancelling here, in particular remember that $x = g(y)$.

$$ Acceptance~probability = \operatorname{min} \left(1, \frac{ f(x_{proposal}) }{ f(x_{current}) } \frac{ \frac{f(x_{current})}{ \left| \frac{d}{dx} g^{-1}(x_{current}) \right| } }{ \frac{f(x_{proposal})}{ \left| \frac{d}{dx} g^{-1}(x_{proposal}) \right| } } \right) $$

$$ Acceptance~probability = \operatorname{min} \left(1, \frac{ { \frac{d}{dx} g^{-1}(x_{proposal}) } }{ { \frac{d}{dx} g^{-1}(x_{current}) } } \right) $$

Finally, I think you can rearrange a little more, and make use of the fact that the derivative of the inverse is the reciprocal of the derivative of the original function:

$$ Acceptance~probability = \operatorname{min} \left(1, \frac{ { \frac{d}{dy} g(y_{current}) } }{ { \frac{d}{dy} g(y_{proposal}) } } \right) $$