Yes, this has been asked before here, but for different reasons. In the E-Step nothing is calculated, we simply define the function, yet once it is defined it is defined once and for all. We could even define it in a way, such that it would not need to be redefined in each iteration by accepting a second parameter. It makes it look like the EM algorithm is a 2 step algorithm, while all it does it maximises the expectation in each iteration. I ask this question because I am not sure if I misunderstood something or if the description and presentation of the algorithm really is that bad or if there is a historical (or other) reason for the chosen description.

  • $\begingroup$ I cannot see how this question extends the one you reference. In what sense is "nothing calculated"? Since you're maximizing an expectation, whether or not you are explicitly calculating the expectation is a matter of the algorithm, but not of the concept. $\endgroup$
    – whuber
    Apr 14 at 13:54
  • $\begingroup$ The one referenced assumed that in the E-Step the maximization is done already. From what I understand in the E step all that is done is that the function to be maximised in the M step is defined. Is that not correct? $\endgroup$
    – timtam
    Apr 14 at 13:56

1 Answer 1


Let's use an example of a Gaussian mixture model. The model describes the distribution of your data in terms of $k$ clusters, such that each cluster is normally distributed. The model is

$$ f(x; \mu, \sigma,\pi) = \sum_{i=1}^K \pi_i\,\mathcal{N}(x\mid\mu_i,\sigma_i) $$

where $\mu = (\mu_1, \mu_2, \dots, \mu_K)$ is the vector or means, $\sigma = (\sigma_1, \sigma_2, \dots, \sigma_K)$ is the vector of variances, and $\pi = (\pi_1, \pi_2, \dots, \pi_K)$ is the vector of mixing proportions such that $\forall_i\,\pi_i \ge 0$ and $\sum_{i=1}^K \pi_i = 1$.

When we want to estimate the parameters of such a model, there is a chicken-and-egg problem: we could easily estimate the $\mu_i, \sigma_i,\pi_i$ per each cluster only if we knew to which cluster each observation belongs, and we could find out to which cluster the observation belongs if we knew the parameters.

Here E-M algorithm comes to play. You start with randomly initialized parameters and then iterate between the two steps. In the E step, you use the parameters to find the probabilities for each of the samples belonging to specific clusters. In the M step, you use those probabilities as weights to estimate and update the parameters. Such iterations are repeated until you don't see that they are changing anymore. In each step, you calculate something that wouldn't be possible to calculate without taking the other step.

  • $\begingroup$ In the E-step we find probabilities? When you say find, you mean calculate? And then again, why is it called expectation step? $\endgroup$
    – timtam
    Apr 14 at 14:33
  • $\begingroup$ @timtam Yes, calculate. For explanation of the E step check see stats.stackexchange.com/questions/139700/… $\endgroup$
    – Tim
    Apr 14 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.