2
$\begingroup$

I'm wondering if the following two formulations of maximum likelihood inference yield the same result.

Let $Z$ be a 0-or-1 latent random variable and $X$ a random variable that depends on $Z$ according to some known conditional probability distribution $p(x|z)$. If we observe a value $x$ but not $z$, then the task of maximum likelihood inference is to find $z^*$ that maximizes $p(x|z^*)$.

Now consider the following alternative formulation. Let $Z'$ be Bernoulli$(\theta)$, where $\theta$ is a parameter, and $X'$ depend on $Z'$ in the same way. In other words, $p(x|z)$ and $p(x'|z')$ are the same conditional distribution. This induces a distribution $p_\theta(x')$ on $x$ given by $p_{\theta}(x') = \sum_z p(x'|z') p_\theta(z')$ where $p_\theta(z')$ is the probability distribution of Bernoulli$(\theta)$, i.e., $p_\theta(1)=\theta$ and $p_\theta(0)=1-\theta$. Now if we observe a value $x$, we can imagine finding $\theta^*$ that maximizes $p_{\theta^*}(x)$.

My question: Is it true that if $z^*=1$ then $\theta^* \ge 1/2$, and if $\theta^* > 1/2$ then $z^*=1$?

In other words, if we use the alternative formulation to form a maximum likelihood estimate of $\theta$, and then use the most likely value of Bernoulli$(\theta)$ as our estimate of $Z$, does this yield the same result as ordinary maximum likelihood inference (the first formulation)? To put it another way, can we do maximum likelihood inference by parametrizing a model for the latent variable we want to infer, forming a maximum likelihood estimate for those parameters, and then using that to find the maximum likelihood value of the latent variable?

$\endgroup$
0
$\begingroup$

For this particular case, yes, it works.

If $p(x|0)<p(x|1)$, then $z^*=1$, and $$p_{\theta^*}(x) = p(x|0) (1-\theta^*) + p(x|1) \theta^* = p(x|0) + (p(x|1)-p(x|0))\theta^*,$$ which is maximized at $\theta^*=1$.

Similarly, if $p(x|0)>p(x|1)$, then $z^*=0$ and $\theta^*=0$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.