I'm learning Bayesian inference by myself and having a difficulty for understanding Gibbs sampling.

From what I understood, Gibbs sampling is to draw samples from a given probability distribution $p(x_1, x_2 | \theta)$ where $\theta$ is parameters for the distribution as $x_1 \sim p(x_1|x_2,\theta)$ and $x_2 \sim p(x_2 | x_1, \theta)$ repeatedly so that we can see the distribution.

But at the same time I see (as far as I read) it's used to estimate parameters of a probability distribution and am not sure I understand it correctly so asking this question here.

Let's say $X = \{x_1, x_2, ..., x_n\}$ are observed data from a distribution and we assume the data were generated by a probability distribution $p(x | \theta_1, \theta_2)$ and also assume we can calculate the likelihood $L = p(\theta_1, \theta_2 | X)$.

In this case is it a valid use case of Gibbs sampling to estimate $\theta_1$ and $\theta_2$ as drawing samples as $\theta_1 \sim p(\theta_1 | \theta_2, X)$ and $\theta_2 \sim p(\theta_2 | \theta_1, X)$?

  • $\begingroup$ Yes, assuming you can sample directly from the full conditionals $p(\theta_1 | \theta_2, X)$.... From a Bayesian perspective, data are fixed and parameters are random, so estimating the parameters of a probability distribution is done by drawing samples from a probability distribution -- the posterior distribution of the parameter values given your prior and your observed data. $\endgroup$ – duckmayr May 16 '19 at 17:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.