It might be some straight forward thing.. But I referred to some threads already over internet to understand what exact does it mean when we use terms "evidence maximization" "type II maximum likelihood" or "maximum marginal likelihood".
All are same, and work with the marginal likelihood. i.e we maximize the denominator in Bayes Theorem. How does it differ from MLE?
Empirical Bayes is a means of using the observed data to compute point estimates of the hyperparameters parametrising your priors. Which only makes sense in context of a hierarchical Bayesian model, where you have hyperparameters which parametrise priors on your model parameters.
Maximum likelihood is a frequentist approach - you compute point estimates of the parameters, and there is no uncertainty being modelled in these parameters through the use of priors, parametrised by hyperparameters, on said parameters.
Consider the Latent Variable Model below, and pay particular attention to the plate (the box):
This indicates that the latent Y is not per x, otherwide the plate should have covered the latent Y as well. I use MLE in this case. However, if there is Y per x, and I only have the observed data, but not the corresponding Y for each observed x, then I have no choice but to marginalize out z and maximize p(x), i.e. the evidence (MLE-II).
Summary: solving the model above, was done via maximzing P(x). In words P(x) is called
Almost invariably, you cannot afford to do MLE-II because the evidence is intractable. This is why MLE-I is more common. Even if latent is not given, people still do MLE-I (this is why EM was invented).
