# What is Type II maximum likelihood?

It might be some straight forward thing.. But I referred to some threads already over internet to understand what exact does it mean when we use terms "evidence maximization" "type II maximum likelihood" or "maximum marginal likelihood".

All are same, and work with the marginal likelihood. i.e we maximize the denominator in Bayes Theorem. How does it differ from MLE?

• Perhaps you might have a look at the following wiki on empirical Bayes which is the same as type-II maximum likelihood, only the latter name is not listed in wiki: en.wikipedia.org/wiki/Empirical_Bayes_method. Mar 20, 2021 at 13:10

Empirical Bayes is a means of using the observed data to compute point estimates of the hyperparameters parametrising your priors. Which only makes sense in context of a hierarchical Bayesian model, where you have hyperparameters which parametrise priors on your model parameters.

Maximum likelihood is a frequentist approach - you compute point estimates of the parameters, and there is no uncertainty being modelled in these parameters through the use of priors, parametrised by hyperparameters, on said parameters.

• Can you say even further that any point estimate is frequentist approach? Mar 23, 2021 at 20:53
• @GoodLuck No, that is an incorrect statement. Mar 27 at 6:31

Consider the Latent Variable Model below, and pay particular attention to the plate (the box):

This indicates that the latent Y is not per x, otherwide the plate should have covered the latent Y as well. I use MLE in this case. However, if there is Y per x, and I only have the observed data, but not the corresponding Y for each observed x, then I have no choice but to marginalize out z and maximize p(x), i.e. the evidence (MLE-II).

Summary: solving the model above, was done via maximzing P(x). In words P(x) is called

• evidence (name stems from Bayes rule)
• Marginal Likelihood (because it is like P(x|z) but z is marginalized out.
• Type || MLE ( to distinguish it from standard MLE where you maximize P(x|z).

Almost invariably, you cannot afford to do MLE-II because the evidence is intractable. This is why MLE-I is more common. Even if latent is not given, people still do MLE-I (this is why EM was invented).