In an image processing class, I dont really get behind the idea how to 'sample from a posterior' with Gibbs sampling. We have a posterior distribution:

$f(z_1, .. ,z_n \mid x_1,.. ,x_n) := f(z \mid x)$

From Bayes theorem we express this as:

$f(z \mid x)$ ∝ $f(x \mid z)f(z)$ ,

where $f(x \mid z)$ is the likelihood and $f(z)$ the prior, which we are modeling in different fashions.

In a first task, we are asked to 'sample from the prior' with Gibbs sampling. To that end, we sample from $f(z)$ by sampling from its conditional distribution by updating the random variables elementwise from $f(z_1 \mid z_2 , .. z_n)$.

Now, we have to 'sample from the complete posterior' and this is where my intuition fails. Iam confused as this is already a conditional distribution. Could anyone explain the general approach of using Gibbs sampling to sample from a posterior distribution?


  • $\begingroup$ Can you clarify what a "complete posterior" distribution is? $\endgroup$ – AdamO Nov 8 '17 at 20:35
  • $\begingroup$ We regard the posterior as f(z∣x). Our task is to use Gibbs sampling to draw samples from that distribution. The word complete might be misleading here. $\endgroup$ – kirtap Nov 8 '17 at 21:09

Typically and in simplest form, Gibbs sampling would be sampling from each of the full conditional distributions

$$f(z_i|\mathbf{z}_{j\neq i},\mathbf{x})$$

using some suitable sampling order (e.g. random scan).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.