I'm dealing with Gibbs Sampling now. Let's consider the example: I know the distribution of X|Y and the distribution of Y. They are some known - Binomial or Beta or other but particular. Thus I have in analytical view f(X|Y), f(Y) and I can calculate joint distribution f(X,Y). To provide Gibbs Sampling I need to calculate many times X∼X|Y and Y∼Y|X. The question is: which technique can I use to sample Y∼Y|X in general case, for any given f(X|Y), f(Y)?


It is not exactly like that. If it is easy and effective to sample from $X\mid Y=y$ and $Y\mid X=x$, then plain vanilla Gibbs sampling is probably the way to go for sampling from the joint distribution of $X,Y$. There may be cases when it is difficult to sample from some of the full conditionals. In those cases, something like a "Metropolis within Gibbs" algorithm may be a good strategy. Take a look at these notes. A suggestion: I believe that you shouldn't study the subject this way. You must abstract the general concepts working with specific concrete examples. Trying to extend those concrete examples will show you where the difficulties arise. If you're looking for a very good book on the subject, check out the latest edition of Robert and Casella.

| cite | improve this answer | |
  • $\begingroup$ But when I do something in OpenBugs, for example, I just need to write {T∼dbeta(3,7); X∼dbin(T,n)} n=15. And this program through Gibbs Sampling automatically sample many times X|T and T|X. It means that they use some numerical technique for sampling T|X cause I can put any distributions and I'll get correct results. How can I do this, sample from T|X, in R, for example? $\endgroup$ – Oleg Mar 26 '13 at 15:07
  • $\begingroup$ We need to know more about OpenBUGS internals to answer that. From their site: "BUGS includes a range of algorithms that its expert system can assign to each such computational task." $\endgroup$ – Zen Mar 26 '13 at 17:43
  • $\begingroup$ I agree that there is a lot it BUGS algorithms but I'm interesting just in sampling from conditional distributions. I don't think, they introduced something new. Perhaps they are using just known technique $\endgroup$ – Oleg Mar 27 '13 at 13:35
  • $\begingroup$ Did you read the notes pointed in my answer? They are probably using "Metropolis within Gibbs", but I can't know that for sure. $\endgroup$ – Zen Mar 27 '13 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.