Seems like full conditionals are often quite difficult to derive, yet programs like JAGS and BUGS derive them automatically. Can someone explain how they algorithmically generate full conditionals for any arbitrary model specification?

  • $\begingroup$ I don't remember exactly, but I know that they often make a lot of very strong assumptions that are often not true. $\endgroup$ – user25658 Aug 23 '13 at 14:50
  • 3
    $\begingroup$ About your comment, what kind of assumptions are you talking about ? $\endgroup$ – Stéphane Laurent Sep 27 '13 at 22:48

Reading through the comments on the othe answers, I believe the correct answer to the question that was intended to be asked is "they don't", in general. As has been mentioned, they construct a DAG and look at the Markov blanket and then (roughly) do the following.

  1. If the Markov blanket around a node correspond to a full conditional that is in a lookup table (say, because it is conjugate) sample from using the technique in the lookup table.
  2. Else, check is the if the unnormalized full conditional density - which is trivial to calculate - is log-concave. If it is, use adaptive rejection sampling.
  3. Else, sample using Metropolis-within-Gibbs to sample from the distribution approximately. While this isn't an exact sample, it can be shown that this algorithm still leaves the posterior invariant.

This isn't exactly what is being done; for example, JAGS will use some other tricks to construct block updates. But this should give an idea of what they are doing.

| cite | improve this answer | |
  • $\begingroup$ I don't know the BUGS and JAGS documentation so well, where is that reported, BTW? $\endgroup$ – altroware Sep 2 '17 at 22:55
  • $\begingroup$ @altroware I don't know the documentation well either, sorry. I know rjags has a function that lists the samples it is using, something like list_samplers but you can check the usual documentation in R to find that. $\endgroup$ – guy Sep 3 '17 at 17:51

I'm not sure why you think full conditionals are hard to derive. Given a complete joint prior probability density $\pi(\cdot)$ for both the parameters and data, the full conditional of, say, $\theta_i$ given $\theta_{-i}$ and the data is easy to derive: it's just proportional to the joint prior for parameters and data. It's easy to tell by inspection which elements of $\theta_{-i}$ can be dropped from the full conditional for $\theta_i$.

Automatic Gibbs samplers carry out this "inspection" by parsing a model specification into a probabilistic directed acyclic graph model. They then compute full conditionals as proportional to the full joint density of $\theta_i$'s Markov blanket.

| cite | improve this answer | |
  • $\begingroup$ Sure it's conceptually simple, but in practice it's not obvious to me that closed form expressions for the full conditionals would be trivial to derive in an algorithmic way. At each layer in the DAG, you can have all sorts of transformations - multiplicative interactions, absolute value, log, square root transformations etc. There are also non-conjugate relationships. $\endgroup$ – user4733 Aug 29 '13 at 17:05
  • $\begingroup$ It's not clear to me how one automatically is able to algorithmically integrate out expressions for the full conditional distributions. Perhaps these programs avoid needing to obtain closed-form expressions for the full conditionals, but I'm trying to get a better understanding of how this gets implemented in practice. $\endgroup$ – user4733 Aug 29 '13 at 17:06
  • $\begingroup$ The normalization constant does not matter, hence one always has closed form expressions because we only multiply closed form expressions. No ? (I'm tired) $\endgroup$ – Stéphane Laurent Sep 27 '13 at 22:46
  • $\begingroup$ It practice it seems quite difficult to me as well for BUGS to determine the conditional distribution. Perhaps providing an example (non-trivial) of the DAG process will help... $\endgroup$ – Glen Sep 28 '13 at 5:15
  • $\begingroup$ @Glen If you provide an example that's given you difficulty, I'll do the inspection. $\endgroup$ – Cyan Sep 28 '13 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.