Gibbs sampling versus general MH-MCMC I have just been doing some reading on Gibbs sampling and Metropolis Hastings algorithm and have a couple of questions. 
As I understand it, in the case of Gibbs sampling, if we have a large multivariate problem, we sample from the conditional distribution i.e. sample one variable while keeping all others fixed whereas in MH, we sample from the full joint distribution.
One thing the document said was that the proposed sample is always accepted in Gibbs Sampling i.e. the proposal acceptance rate is always 1. To me this seems like a big advantage as for large multivariate problems it seems that the rejection rate for MH algorithm becomes quite large. If that is indeed the case, what is the reason behind not using Gibbs Sampler all the time for generating the posterior distribution?
 A: Gibbs sampling breaks the curse of dimensionalality in sampling since you've broken down the (possibly high dimensional) parameter space into several low dimensional steps. Metropolis-Hastings alleviates some of the dimensionaltiy problems of generate rejection sampling techinques, but you are still sampling from a full multi-variate distribution (and deciding to accept/reject the sample) which causes the algorithm to suffer from the curse of dimensionality.
Think of it in this simplified way: it is much easier to propose an update for one variable at a time (Gibbs) than all variables simultaneously (Metropolis Hastings). 
With that being said, the dimensionality of the parameters space will still affect convergence in both Gibbs and Metropolis Hastings since there are more parameters that could potentially not converge. 
Gibbs is also nice because each step of the Gibbs loop may be in closed form. This is often the case in hierarchical models where each parameter is conditioned on only a few others. It is often pretty simple to contstruct your model so that each Gibbs step is in closed form (when each step is conjugate it's sometimes called "semi-conjugate"). This is nice because you're sampling from known distributions which can often be very fast. 
A: the main rationale behind using the Metropolis-algorithm lies in the fact that you can use it even when the resulting posterior is unknown. For Gibbs-sampling you have to know the posterior-distributions which you draw variates from.
