6
$\begingroup$

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one prefer either of them?

$\endgroup$
1
  • $\begingroup$ Why can't you directly compute the Maximum Likehood by setting the derivative to 0 and solving an equation system, but have to do it iteratively with EM? $\endgroup$ Oct 11 '17 at 18:22
2
$\begingroup$

You can use ML directly but as the priors of the different gaussians (usually called latent variables) are unknown, you'll probably find that your optimization objective is pretty hard. EM iterative method solves this intractability.

Suggested read: https://see.stanford.edu/materials/aimlcs229/cs229-notes8.pdf

$\endgroup$
4
  • $\begingroup$ So, EM is almost always used in this case? $\endgroup$
    – Mehrin
    Oct 14 '16 at 11:20
  • $\begingroup$ Yes, and as the number of gaussians in the mixture model grows, the probability of not using EM is reduced even further. $\endgroup$
    – Gabizon
    Oct 14 '16 at 11:23
  • $\begingroup$ And do you know why gradient is used in Fisher vectors? Does it have to do with MLE? $\endgroup$
    – Mehrin
    Oct 14 '16 at 11:27
  • $\begingroup$ This question is not related to your original question. Mark this question answered, ask another one and I'll answer there. $\endgroup$
    – Gabizon
    Oct 14 '16 at 11:28
9
$\begingroup$

Mehrin, you are in danger of creating a false dichotomy. EM is an optimisation technique that can be used to find maximum likelihood estimates and so the choice is not "one or the other".

In mixture models EM is often used to find MLEs or MAPs as it produces transparent algorithms.

$\endgroup$
2
  • 2
    $\begingroup$ You can, however, use MLE directly i.e. take the gradient of pdf w.r.t. the mixture parameters and use gradient ascent. $\endgroup$
    – Mehrin
    Oct 14 '16 at 10:46
  • 2
    $\begingroup$ @Mehrin, yes, but the question title should probably be changed to "direct optimization versus expectation-maximization algorithm." EM is an implementation of MLE. $\endgroup$
    – Adrian
    Oct 11 '17 at 19:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.