I'm starting to learn about Gibbs sampling, having so far only worked with Metropolis-Hastings MCMC, and there's something I haven't grasped yet about the way Gibbs sampling works.

To frame the problem, consider the following posterior distribution:

$$ P(\underline{\theta} | \underline{y}) \propto \exp{\left[\sum_i \left( y_i \ln{F(x_i , \underline{\theta})} - F(x_i , \underline{\theta}) \right) \right]} $$ This is a product of Poisson distributions, and $F(x , \underline{\theta})$ is some known, calculable, non-linear function which cannot be expressed as a product over functions of single parameters, i.e. $$ F(x, \underline{\theta}) \ne f_1(x, \theta_1) f_2(x, \theta_2) \dots f_n(x, \theta_n)$$ This fairly typical of a posterior you might find doing physics data analysis, for example.

Is it possible (or appropriate) to use Gibbs sampling with such a posterior?

The univariate conditional distributions for each parameter can be calculated, but only by evaluating the whole posterior and fixing the appropriate parameters - they do not have separate closed forms.



1 Answer 1


If you can write the joint density $p(\theta|y)$ in closed form, you can deduce $$p(\theta_i|y,\theta_{-i})\propto p(\theta|y)$$ in closed form. At this stage, the practical possibility in using Gibbs depends on the complexity of the above density and on the computing time requirements.

Note that Gibbs sampling is a specific type of Metropolis-Hastings algorithm. There is nothing magical about it, from the probability $1$ of acceptance to the use of the proper conditional densities.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.