Consider the Latent Variable Model below, and pay particular attention to the plate (the box):
This indicates that the latent Y is not per x, otherwide the plate should have covered the latent Y as well. I use MLE in this case. However, if there is Y per x, and I only have the observed data, but not the corresponding Y for each observed x, then I have no choice but to marginalize out z and maximize p(x), i.e. the evidence (MLE-II).
Summary: solving the model above, was done via maximzing P(x). In words P(x) is called
- evidence (name stems from Bayes rule)
- Marginal Likelihood (because it is like P(x|z) but z is marginalized out.
- Type || MLE ( to distinguish it from standard MLE where you maximize P(x|z).
Almost invariably, you cannot afford to do MLE-II because the evidence is intractable. This is why MLE-I is more common. Even if latent is not given, people still do MLE-I (this is why EM was invented).