What is the $i$th sufficient statistic in the EM algorithm for Gaussian mixture models? I am reading up on the EM algorithm for Gaussian Mixture Models, and there is consistent reference to the $i$th sufficient statistic.  What is this, and why is it relevant to the algorithm?
 A: This question concerns a lecture on EM with an exponential family of distributions.  The logarithm of each density $f$ in this family can be expressed (up to a normalization constant) as
$$\log(f_X(x|\mathbf{\theta})) = \eta(x) + \sum_i s_i(x) \theta_i$$
where $\mathbf{\theta} = (\theta_i)$ is a vector of parameters.  Apart from influencing the normalization constant (which doesn't matter for inference), parameter $\theta_i$ enters the formula solely through its product with $s_i(x)$, a function of the observation $x$.  Thus, everything that can be known about $f$ depends on $x$ only through the vector of values $\mathbf(s) = (s_i(x))$.  It is a short step to conclude that the functions $s_i$ generate a set of sufficient statistics for this family.  The lecture uses the term "ith sufficient statistic" to refer to $s_i$.
This form of distribution function is one of the simplest possible in any theory or algorithm, like EM, that relies on the log likelihood function, because the log likelihood is a $\theta$-linear combination of the sufficient statistics: the linearity implies the log-likelihood is a differentiable function of the parameters and the partial derivatives can be instantly recognized (they are obtained by summing each $s_i(x)$ over the dataset).  Thus it leads to relatively simple formulas and a mathematically tractable analysis.
