When I learned EM algorithm, I saw many literatures use (the expectation of ) the log-likelihood. Is there any reason other than that the log-likelihood may reduce computation?


  • 3
    $\begingroup$ It's worth pointing out that reducing computational burden is actually an important feature, because original-scale likelihoods can produce very small decimals. It's not just about making computation more time- and memory-efficient; it's about producing fewer underflow and roundoff errors that can have very real effects on estimation, especially when working with iterative algorithms like EM. Not to mention it makes the analytical math easier. $\endgroup$ Feb 8 '15 at 16:31
  • 1
    $\begingroup$ The fundamental reason is that, if you apply the EM algorithm to the likelihood itself, the proof of convergence to a local maximum, based on the Jensen inequality, does not work. $\endgroup$
    – Xi'an
    Feb 8 '15 at 17:28

Generally, statisticians like to take the log-likelihood because it is easier to maximize. If we find the value of $x$ that maximizes $\log(f(x))$ we've also found the value of $x$ that maximizes $f(x)$ due to the monotonicity of logarithms. This is also what is happening in the EM algorithm.

The reason it is easier to maximize is because of the nice property of logs that says $\log(f(X)g(Y)h(Z)) = \log(f(X)) + \log(g(Y)) + \log(h(Z))$. When maximizing, a common approach is to use partial derivatives, set it equal to zero, and solve for the variable we are interested in. It is much simpler and easier to take the partial derivative of a sum of functions than a product of functions. The log function takes a product, $XYZ$, and turns it into a sum $\log(X) + \log(Y) + \log(Z)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.