# How to understand the binary latent variable z in GMM model?

GMM(Gaussian Mixture Model) itself is a mixture of Gaussian with each having the proportion of $$\pi_k$$, $$\sum_{k=1}^{K}\pi_k=1$$this is easy to understand. But when introducing the latent, I don't understand it, because there can only be one $$z_k=1$$, then how this is related to the soft assignment in GMM model?

It may help if you can define the $$z$$ variable a bit more thoroughly in your post, since people can use this variable for different things.

However I am going to assume that you are referring to $$z$$ as a latent variable that randomly selects from a one-hot encoded vector which Gaussian to choose, for example when randomly sampling from the GMM.

For example, when randomly sampling a GMM you need to pick "one out of $$K$$" Gaussians to pull the sample from. How do you automate that process? You define an extra latent variable, $$z$$ to act as the middle man in a sense which communicate which Gaussian to pick from a one hot encoded vector of choices.

So for some data point, $$X_i$$, we can ask the question what is the probability distribution of $$X_i$$ assuming it belongs to class "2", so it allows to formalize questions such as: $$P(X_i | Z = 2)?$$, to which we can allocate its Gaussian $$P(X_i | Z = 2) = \mathcal{N}(\mu_2,\sigma^2_2)$$, and then we can assign multinomial priors to each cluster and even as $$P(Z=2)$$, what is the prior that we belong to class 2. And so on.

So it's a latent middle man that helps with the random choice of which Gaussian to pick in the mixture when sampling, or calculting the EM algorithm etc.

• Thank you for your answer, it is very helpful. I have another question, in M-step, the GMM is maximizing the log likelihood of complete data $argmax_{\theta}\int p(z|x,\theta^{old})lnp(x,z|\theta)dz$, is is universal for all EM problems even if it is not gaussian mixture models? May 26 at 7:27