I would like to sample from: $$ p(\theta_2|x)=\int p(\theta_2|\theta_1,x) . p(\theta_1|x) . d\theta_1 $$

Several persons suggest me to use the following procedure to draw $(\theta_2^i)_{i=1:N}$:

for i=1:N
    draw $\theta_1^i$ from $p(\theta_1|x)$
    draw $\theta_2^i$ from $p(\theta_2|\theta_1^i,x)$

this appears intuitively satifying but I do not see how to show that this procedure is valid. It appears to be linked to Rao-Blackwellisation but it is not clear to me. Thanks for your help.

(this question is linked to an older one that I posted Sampling from marginal using integrated conditional)


1 Answer 1


No need to invoke Rao-Blackwellization; it's just straight probability.

Let's take conditioning on $x$ everywhere as given.

You know $p(\theta_2,\theta_1) = p(\theta_2|\theta_1)\cdot p(\theta_1)$; the sampling scheme just samples from the LHS by sampling from the scheme on the RHS - sample from $p(\theta_1)$ and then from $p(\theta_2|\theta_1)$.

When you do the first step, $\theta_1$ has the density $p(\theta_1)$. When you do the second step, conditionally, $\theta_2$ is from $p(\theta_2|\theta_1)$, but since $\theta_1$ is from its marginal, repeated sampling of the $(\theta_1,\theta_2)$ pairs is from the joint distribution, and hence the $\theta_2$ values considered alone are from their margin.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.