4
$\begingroup$

I my opinion, EM algorithm is used to estimate the parameters of some complex log likelihood function. Because sometimes, it's hard to get the derivative, we can use EM algorithm. But if we have some auto differentiate tools such as theano, we can compute the derivative automatically. Can these tools replace EM algorithm?

$\endgroup$
4
$\begingroup$

In some cases yes; autodiff certainly makes life easier in many circumstances. But the EM algorithm may still be more appropriate in other cases. For example, consider fitting mixture models. At each step, the EM algorithm will always return valid parameters for the distribution. Although gradient-based optimization methods can be used, complicated constraints may be necessary. The mixture weights must be constrained to be positive and sum to one, covariance matrices must be constrained to be positive semidefinite, etc. The EM algorithm may have better convergence for some (but certainly not all) problems.

The following posts may also of interest: 1, 2, 3.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.