1
$\begingroup$

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of GMMs and how it is a linear superposition of Gaussians in the form $p(\textbf{x}) = \sum\limits_{k=1}^K \pi_k\mathcal{N}(\textbf{x}\vert\mu_k,\Sigma_k)$, but I don't understand what purpose the latent variable serves in this and why a joint distribution needs to be defined in terms of the marginal $p(\textbf{z})$ and $p(\textbf{x}\vert\textbf{z})$. Is there an intuitive reason for the variable $\textbf{z}$?

$\endgroup$
2
$\begingroup$

The latent variable $\mathbf{z}$ is the allocation vector that attributes to each observation $x_i$ of the sample its component indicator $z_i$, i.e.,$$\mathbb{P}(Z_i=\xi)=p_\xi\qquad\qquad X_i|Z_i=\xi \sim \mathcal{N}(\mu_\xi,\Sigma_\xi)$$The latent variable is used in the EM algorithm and the Gibbs sampler. It is also a key notion behind clustering & classification.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.