Maximum likelihood inference by estimating the parameters of the probability distribution

I'm wondering if the following two formulations of maximum likelihood inference yield the same result.

Let $$Z$$ be a 0-or-1 latent random variable and $$X$$ a random variable that depends on $$Z$$ according to some known conditional probability distribution $$p(x|z)$$. If we observe a value $$x$$ but not $$z$$, then the task of maximum likelihood inference is to find $$z^*$$ that maximizes $$p(x|z^*)$$.

Now consider the following alternative formulation. Let $$Z'$$ be Bernoulli$$(\theta)$$, where $$\theta$$ is a parameter, and $$X'$$ depend on $$Z'$$ in the same way. In other words, $$p(x|z)$$ and $$p(x'|z')$$ are the same conditional distribution. This induces a distribution $$p_\theta(x')$$ on $$x$$ given by $$p_{\theta}(x') = \sum_z p(x'|z') p_\theta(z')$$ where $$p_\theta(z')$$ is the probability distribution of Bernoulli$$(\theta)$$, i.e., $$p_\theta(1)=\theta$$ and $$p_\theta(0)=1-\theta$$. Now if we observe a value $$x$$, we can imagine finding $$\theta^*$$ that maximizes $$p_{\theta^*}(x)$$.

My question: Is it true that if $$z^*=1$$ then $$\theta^* \ge 1/2$$, and if $$\theta^* > 1/2$$ then $$z^*=1$$?

In other words, if we use the alternative formulation to form a maximum likelihood estimate of $$\theta$$, and then use the most likely value of Bernoulli$$(\theta)$$ as our estimate of $$Z$$, does this yield the same result as ordinary maximum likelihood inference (the first formulation)? To put it another way, can we do maximum likelihood inference by parametrizing a model for the latent variable we want to infer, forming a maximum likelihood estimate for those parameters, and then using that to find the maximum likelihood value of the latent variable?

1 Answer

For this particular case, yes, it works.

If $$p(x|0), then $$z^*=1$$, and $$p_{\theta^*}(x) = p(x|0) (1-\theta^*) + p(x|1) \theta^* = p(x|0) + (p(x|1)-p(x|0))\theta^*,$$ which is maximized at $$\theta^*=1$$.

Similarly, if $$p(x|0)>p(x|1)$$, then $$z^*=0$$ and $$\theta^*=0$$.