I'm trying to use Gibbs sampling to get the following target distribution:

$$ p(a,b,c \lvert x, z) $$

Where $z = f(x,a,b,c)$ and the rest are independent. I know the following conditional probabilities and can sample from them:

$$ p(a\lvert x, z, b, c), \, p(b\lvert x, z, a, c), \, p(c\lvert x, z, b, a)$$

My question is how to express the target distribution I need using the ones above.

My attempt was to start with the following:

$$ p(a,b,c \lvert x, z) = \frac{p(a,b,c,x,z)}{p(x,z)} $$

But I'm not sure how to continue (or if it's the right direction)


1 Answer 1


This is the Hammersley-Clifford(-Besag) theorem:. Here is the version presented in our book.

Definition: Let $(Y_1, Y_2, \ldots, Y_p) \sim g(y_1,\ldots,y_p)$, where $g^{(i)}$ denotes the marginal distribution of $Y_i$. If $g^{(i)}(y_i)> 0$ for every $i=1, \ldots, p$, implies that $$g(y_1,\ldots,y_p) > 0$$ then $g$ is said to satisfy the positivity condition.

Theorem (Besag, 1974) Under the above positivity condition, the joint distribution $g$ satisfies $$ g(y_1,\ldots,y_p) \propto \prod_{j=1}^p \; {g_{\ell_j}(y_{\ell_j}|y_{\ell_1}, \ldots,y_{\ell_{j-1}},y_{\ell_{j+1}}^\prime,\ldots,y_{\ell_p}^\prime) \over g_{\ell_j}(y_{\ell_j}^\prime|y_{\ell_1},\ldots,y_{\ell_{j-1}}, y_{\ell_{j+1}}^\prime,\ldots,y_{\ell_p}^\prime)} $$ for every permutation $\ell$ on $\{1,2,\ldots,p\}$ and every $y' \in \mathfrak Y$.

The conditioning on $(x,z)$ mentioned in the question is irrelevant in that case, since every distribution appearing above is then conditional on $(x,z)$.

  • 1
    $\begingroup$ I really appreciate the answer, but I'm not sure how to use the theorem for my case yet $\endgroup$ Mar 19, 2021 at 18:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.