# Gaussian mixture distribution confusion

Have some confusion on Gaussian mixture distribution model (reference is from the book of Pattern Recognition and Machine Learning). My confusion is how below formula works?

$\sum_kz_k=1$

My thought is, $z_1$ could be $0$ or $1$, $z_2$ could be $0$ or $1$, ... $z_K$ could be $0$ or $1$, how could sum of them (them I mean $z_1$, $z_2$, ..., $z_K$) to be 1?

• $z_k$ is a binary component of a $K$ dimensional vector, $z$. The purpose of $z$ is to indicate which one of the $K$ populations a particular observation comes from. Under the model, an observational element must belong to one and only one of the $K$ populations. Hence the summation to 1. – lmo Nov 25 '16 at 20:39
• Imo is correct but to complete the argument the probability that zk is 1 is the proportion that corresponds to that specific normal distribution in the mixture. – Michael Chernick Nov 25 '16 at 23:29
• "binary component" means that it is binary, either 0 or 1, and that it is one element in the vector. It is more or less a combination of two math terms, but probably with a computer science influence. To emphasize, let's say $z = (0, 1, 0, 0, 0)$. Then $z_1=0$, $z_2=1$, and $z_k=0$ for $k \in \{1, 3, 4, 5 \}$. – lmo Nov 26 '16 at 13:01
• That is correct. – lmo Nov 27 '16 at 13:42
• just fyi it's usually called the one-hot representation – dontloo Dec 1 '16 at 9:18

For each $x_i$ in the sample $(x_1,\ldots,x_n)$ one can create an binary (and unobserved) vector $\mathbf{z}_i=(z_{i1},\ldots,z_{iK})$ made of components in $\{0,1\}$ such that one and only one of the $z_{ij}$'s is equal to one, e.g., $\mathbf{z}_i=(0,\ldots,0,1,0,\ldots,0)$, and all others are equal to zero. The corresponding $j$ is called the component of $x_i$. Conditional on $\mathbf{z}_i$, $x_i$ is distributed as a normal variate $$\mathcal{N}\left(\sum_{j=1}^K z_{ij}\mu_j,\sum_{j=1}^K z_{ij}\Sigma_j\right)$$Marginally, the vector $\mathbf{z}_i$ is distributed as a Multinomial $\mathcal{M}_K(1;\pi_1,\ldots,\pi_k)$ variate.
• Thanks Xi'an, vote up, what means $\mu_j$ and $\sum_j$ in your formula? – Lin Ma Dec 1 '16 at 7:32