5
$\begingroup$

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one prefer either of them?

$\endgroup$
  • $\begingroup$ Why can't you directly compute the Maximum Likehood by setting the derivative to 0 and solving an equation system, but have to do it iteratively with EM? $\endgroup$ – hypnoticpoisons Oct 11 '17 at 18:22
2
$\begingroup$

You can use ML directly but as the priors of the different gaussians (usually called latent variables) are unknown, you'll probably find that your optimization objective is pretty hard. EM iterative method solves this intractability.

Suggested read: https://see.stanford.edu/materials/aimlcs229/cs229-notes8.pdf

$\endgroup$
  • $\begingroup$ So, EM is almost always used in this case? $\endgroup$ – Mehrin Oct 14 '16 at 11:20
  • $\begingroup$ Yes, and as the number of gaussians in the mixture model grows, the probability of not using EM is reduced even further. $\endgroup$ – Gabizon Oct 14 '16 at 11:23
  • $\begingroup$ And do you know why gradient is used in Fisher vectors? Does it have to do with MLE? $\endgroup$ – Mehrin Oct 14 '16 at 11:27
  • $\begingroup$ This question is not related to your original question. Mark this question answered, ask another one and I'll answer there. $\endgroup$ – Gabizon Oct 14 '16 at 11:28
8
$\begingroup$

Mehrin, you are in danger of creating a false dichotomy. EM is an optimisation technique that can be used to find maximum likelihood estimates and so the choice is not "one or the other".

In mixture models EM is often used to find MLEs or MAPs as it produces transparent algorithms.

$\endgroup$
  • 1
    $\begingroup$ You can, however, use MLE directly i.e. take the gradient of pdf w.r.t. the mixture parameters and use gradient ascent. $\endgroup$ – Mehrin Oct 14 '16 at 10:46
  • 2
    $\begingroup$ @Mehrin, yes, but the question title should probably be changed to "direct optimization versus expectation-maximization algorithm." EM is an implementation of MLE. $\endgroup$ – Adrian Oct 11 '17 at 19:59

protected by kjetil b halvorsen Oct 11 '17 at 18:46

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.