Why Gibbs Sampling for mixture models?

Question

I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its convenience relies in the simplicity of sampling from the full conditionals for the parameters, however I was wondering whether and why this method should be preferable to others that were introduced in previous chapters and with which I am less familiar (like Variational methods, or others I don't know).

Furthermore, the book starts by explicitly writing the full joint $p(x,z,\mu,\Sigma, \pi)$ for the gmm (assuming semi-conjugate prior), am I right to say that even knowing this explicitly, sampling directly (using for instance rejection sampling) is inefficient because of the curse of dimensionality, while Gibbs suffer far less from this phenomenon? Is this the only reason why Gibbs is to be preferred?

Any help on understanding pros and cons of the various existing methods for inferring the posterior is well accepted.

Answer 1

Gibbs sampling is possibly the first MCMC algorithm implemented for mixture models (Gelber, Gelman and Goldhirsch, 1989), inspired from the data augmentation of Tanner and Wong (1987) and ultimately from the EM algorithm. It takes advantage of the latent variable structure in producing nice, low dimension, and natural conditionals that can be simulated quite efficiently, much more than a default MCMC algorithm like random walk Metropolis-Hastings.

However, here is a slide from my MCMC course, where I illustrate the potential pitfall of using plain Gibbs for a mixture of two Gaussians with unknown means, namely that it may get trapped in a local mode because the latent variables are practically if not theoretically stuck at a fixed value.