I am new to Gibbs Sampling and I do understand how the algorithm works but I would also like to understand how sampling from the conditional distributions is equivalent to sampling from the joint.

Given a joint distribution $f(\theta_1,\theta_2)$ with parameters $\theta_1$ and $\theta_2$, gibbs sampling samples $\theta$ from $\theta_1 \sim f(\theta_1|\theta_2)$ and $\theta_2 \sim f(\theta_2|\theta_1)$.

When we sample $\theta_1, \theta_2$ however, we are actually sampling from the joint distribution $f(\theta_1,\theta_2^{(t-1)})$ when sampling $\theta_1$ and $f(\theta_2^{(t)},\theta_1)$. This is equivalent to sampling from the conditionals $f(\theta_1|\theta_2^{(t-1)})$ and $f(\theta_2|\theta_1^{(t)})$.

How is this equivalent to samples coming from a joint distribution $f(\theta_1,\theta_2)$ ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.