I saw in many articles that EM is an algorithm to do MLE, and we usually use it when a direct MLE is not possible.

Can someone tell me what is the meaning of "direct MLE is not possible" (and what are the reasons for not being possible)? Preferably with an example.


As an example, let us assume you have a Normal distribution that is censored above some value $\theta$, so that for a given sample you know how many observations are greater than $\theta$ but not what their values are. (You do know the values of the observations that are less than $\theta$.) If you want the maximum likelihood estimators of the parameters $\mu$ and $\sigma^2$ of the distribution, you'll have to resort to some iterative procedure, because there is no closed-form solution (so the "direct MLE is not possible") to the problem of finding the estimates. (In the uncensored case, you calculate the MLEs using the sample mean and sample variance with denominator equal to the sample size.) EM is one such iterative algorithm, and a widely-applicable one at that.

| cite | improve this answer | |
  • $\begingroup$ Thanks for answering, but actually I did not understand what you are trying to say for many reasons. First what do you mean by "censored"? Second you said "there is no closed-form solution" and this is my whole question about, why "there is no closed-form solution"? Could you explain this "mathematically" and in more details why if we take that normal distribution along with the definition of MLE then there will be no closed form solution and also could you explain "mathematically" how EM will help in this case. I appreciate your help and thanks again $\endgroup$ – Mosab Shaheen Apr 4 '19 at 18:29
  • 1
    $\begingroup$ Do you know what a closed-form solution is? $\endgroup$ – jbowman Apr 4 '19 at 18:51
  • $\begingroup$ "Censoring" is explained in the last half of the first sentence, at least "censored above" (censored below works the same way, except with "less than" instead of "greater than".) The word itself is just a quick way of describing that sort of situation. $\endgroup$ – jbowman Apr 4 '19 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.