Which metric is used in the EM algorithm for GMM training ?

Question

My question concerns the expectation-maximisation algorithm used to estimate the hyper-parameters of a Gaussian mixture model in z multivariate setup. I understand that the EM algorithm uses the Maximum likelihood criterion as starting point, but I was wondering about the metric used (implicitly) in such technique.

For different clustering techniques like "Hierarchical clustering" or "k-means", a distance is used (either Euclidian or a different one) to assess the "closeness" of two points; I get that learning a GMM aims at maximizing the likelihood between the model and the training data, but when looked at as an unsupervised clustering technique, is it possible to associate a "hidden" metric to the EM procedure ? And if the answer is yes, could it be changed (maximizing likelihood while taking into account a certain notion of neigborhood between features) ?

Thanks

Answer 1

Just some extension to Anony-Mousse's answer.

K-means is a particular case of EM for GMM that assumes
1) a hard assignment of data points to clusters (the hidden variables), and
2) that the identity matrix is used as the covariance.

Multivariate Gaussian with identity covariance matrix becomes $$\frac{1}{(2\pi\epsilon)^{1/2}}exp\{-\frac{1}{2\epsilon}\lVert x-\mu\rVert^2 \},$$ so minimizing the log likelihood is equivalent to minimizing the squared Euclidean distance.

If we use an arbitrary covariance matrix, the exponential term of Gaussian $$-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu)$$ can be seen as the squared Mahalanobis distance.

The distribution will be no longer Gaussian if the metric is changed.

Answer 2

The "hidden" metric is not at all hidden. It is the Mahalanobis distance.