2
$\begingroup$

I am trying to grasp what exactly is "estimated" in the E-step of the algorithm.

According to all definitions, in E-step the "conditional expectation values , or posterior probabilities of the hidden variables" are computed, using the Bayes formula ( posterior probability=prior( or marginal prob.) x likelihood/probability of evidence). Now, my question is, are these posterior probabilities, that I "estimate" or "compute", actually numbers or are they functions with respect to the current iteration's parameters ( and they are the ones I need to plug into M-step)? Where the probability of evidence would be likelihood, calculated with the previous argmax , calculated in previous M-step.

$\endgroup$
  • $\begingroup$ I would like to know why my question got downvoted, in order to be able to ask a better question later. I have been trying to understand a concept that is difficult for me, and described the issue in my own terms. I was able to receive a sufficient answer for my question. So where is the problem? $\endgroup$ – zima Sep 2 '13 at 10:38
3
$\begingroup$

The 'E' is typically referred to as the 'expectation' step. It refers to the fact that with fixed $\theta$ and fixed data, one can (usually) find the conditional expectation for the hidden states conditional on the current value of the parameters. In step 2, you consider the hidden states fixed (and data of course) and update the current value of the parameters by maximizing the conditional likelihood. And you repeat until the parameters converge. So to answer your question, the expectation step considers the parameters fixed at a particular value, in which case the hidden states are conditionally computable as a function of the current parameters. In the Bayesian context, this value would form a member of the posterior probability density.

Note, however, that the traditional EM algorithm is really built to find the mode (MLE or MAP), and does so by hunting around the posterior. However, it's not fully Bayesian as you do not search, nor try to search, through the entire posterior space and don't attain a posterior PDF as a result. The Bayesian extension is known as variational Bayes.

$\endgroup$
0
$\begingroup$

It's similar to EM used for ML, you can take a look at Section 1.3 from this handout.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.