# Gibbs sampling for Multivariate: how to update?

In this page of Murphy's 'Machine Learning: a Probabilistic Perspective' it's explained how to do Gibbs sampling on a Gaussian Mixture Model.

Reading this, I was trying to understand when to update parameters 'all together' and when to separate them: in Gibbs Sampling, you update one parameter at the time. According to this book, however, you update the 'multidimensional' means for each cluster 'all together'. That is, for each cluster k, it updates $$[\mu_{k1}, \mu_{k2}]$$ in one go. However, would it work if hypothetically I first update $$\mu_{k1}$$ and then $$\mu_{k2}$$?

I am asking to improve my understanding of this method. Thanks

is not correct. Gibbs sampling is using a decomposition of the distribution of interest into conditional distributions for blocks of components of the vector $$X$$ to be simulated. These blocks can be of any dimension. Provided all components of $$X$$ belong to at least one of the blocks, and that there is no irreducibility issue with the conditionals, the associated Gibbs sampler is valid. Hence, if instead of simulating the vector $$\boldsymbol \mu_k$$ at once, as in (24.12), one simulates each component of $$\boldsymbol \mu_k$$ sequentially, conditional on everything else, this is also a correct implementation of the Gibbs sampler.