Two-step maximum likelihood inference

Question

Suppose we have an latent r.v. $Z$ (not observed) and an observed r.v. $X$, where $X$ depends on $Z$ via some conditional distribution $p(x|z)$. Given $x$, we will try to infer $z$.

Standard maximum likelihood inference asks: given $x$, find $z^*$ that maximizes $p(x|z^*)$.

Consider the following alternate "variational" method: we find the distribution $p^*(z)$ that maximizes $\sum_z p(x|z) p^*(z)$, then find the $z^*$ that maximizes $p^*(z^*)$.

Do these two methods always yield the same result $z^*$?

Answer 1

Yes. If $z^*$ maximizes $p(x|z^*)$, then the optimal distribution for $p^*(z)$ is a one-hot distribution that assigns all of its weight to $z^*$, i.e., $p^*(z^*)=1$ and $p^*(z)=0$ for all $z \ne z^*$.