Confusion about the MLE vs EM algorithm From what I understand, the Maximum Likelihood estimate is an formulation of a optimization problem that we want to solve. That is, we want to find the parameters of a distribution which maximizes the likelihood or log-likelihood. Source
There are various approaches for finding the MLE:


*

*EM Algorithm (suffers from local optima)

*Genetic Algorithm (takes tons of time)


Is the MLE just an optimization problem that we are trying to solve? Why do people refer to the MLE as a "method"? I read some people saying: Let's use the MLE method to find the parameters.
From what I understand, the MLE is simply an optimization problem. Is this understanding correct?
 A: Solving the MLE objective function is an optimization problem, but the choice of actually using the MLE to estimate parameter values is a method. For example, to estimate the mean parameter in the Normal distribution, one could use the method of MLE or could use method of moments. Once you choose that you are going to use the MLE estimator, obtaining the estimator is an optimization problem.
A: MLE is not a method. It corresponds to the choice of a score function over the set of distributions you search into. MLE means you take, for your score, the log-likelihood of your distribution given the dataset, that is the probability of the dataset given the distribution : $P(D/\theta)$. So a distribution that will make your dataset more probable than an other will have a higher score.
The problem of MLE is that you do not take into account the prior within your distributions space. For instance, if you toss a dice a get a 3, MLE score will attribute the highest score to the distribution that has a probability of 1 on the 3 output and 0 to the others (which is counter-intuitive).
You can introduce a prior to obtain a more sophisticated score. In that case your score will have two elements : one due to the prior and the other to the likelihood. There are many choices of prior depending on the family of model you want to learn (Bayesian network, Markov network,...)
A: MLE is not just an optimization problem. There is a long scholastic tradition that asserts that the physical world is not random, but it follows some laws. The principle is referred as the "Principle of Sufficient Reason", and other names. If one accepts probabilistic models for events, then, in case an even can have more than one possible cause, more often than not, the most likely cause is is the true cause (which is a tautology, otherwise the most likely cause would not be that most likely). The MLE is a formalization of that principle, but it is more than that, because it can be proved that, under general conditions that cover most applications of probabilistic models, the MLE of parameters are a consistent estimator of the true parameter.    
The EM algorithm is an algorithm that applies when the likelihood function can be written as an expected value over un-observed values (a mixture of distributions). It often simplifies the computational complexity of a direct solution of the MLE problem. However, given the advances in computational power since 19977 (when the EM algorithm was formally introduced), a direct approach in the case of finite mixtures might be competitive. 
