# Gaussian Mixture Models: Maximum Likelihood Estimation or Expectation Maximization?

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one prefer either of them?

• Why can't you directly compute the Maximum Likehood by setting the derivative to 0 and solving an equation system, but have to do it iteratively with EM? – hypnoticpoisons Oct 11 '17 at 18:22

## 2 Answers

You can use ML directly but as the priors of the different gaussians (usually called latent variables) are unknown, you'll probably find that your optimization objective is pretty hard. EM iterative method solves this intractability.

• So, EM is almost always used in this case? – Mehrin Oct 14 '16 at 11:20
• Yes, and as the number of gaussians in the mixture model grows, the probability of not using EM is reduced even further. – Gabizon Oct 14 '16 at 11:23
• And do you know why gradient is used in Fisher vectors? Does it have to do with MLE? – Mehrin Oct 14 '16 at 11:27
• This question is not related to your original question. Mark this question answered, ask another one and I'll answer there. – Gabizon Oct 14 '16 at 11:28

Mehrin, you are in danger of creating a false dichotomy. EM is an optimisation technique that can be used to find maximum likelihood estimates and so the choice is not "one or the other".

In mixture models EM is often used to find MLEs or MAPs as it produces transparent algorithms.

• You can, however, use MLE directly i.e. take the gradient of pdf w.r.t. the mixture parameters and use gradient ascent. – Mehrin Oct 14 '16 at 10:46
• @Mehrin, yes, but the question title should probably be changed to "direct optimization versus expectation-maximization algorithm." EM is an implementation of MLE. – Adrian Oct 11 '17 at 19:59