I am familiar with regression linear models, and EM algorithms. However, I do not get the idea of fitting the mixture of regression linear models using the EM algorithm. So, what I think about it is as follows:

  1. fit the first linear regression model and then estimate the coefficient. Then, find the density of the fitted model. I am confused about this part, as there is no density for the linear regression!
  2. Repeat the first step with the second regression model.
  3. Run EM algorithm.

Is that correct? Could someone help me with an example and manual implementation, please? I knew that there are some R packages, but I would like to understand the implementation manually.

  • 2
    $\begingroup$ conceptually, it might help for you to start by separating out the E step from the M step, and determining what should go in each. $\endgroup$ Commented Jul 27, 2022 at 21:13

1 Answer 1


No, the procedure is wrong.

If you're familiar with the Gaussian mixture, think of the mixture of regression as a Gaussian mixture, while the mean $\mu_k$ now has been substituted by $x_i^\top\beta_k$. Since when we talk about regression, we are talking about the mean regression.

To manually implement the algorithm, you should know how to derive the E-step and the M-step based on clear concepts of observed log-likelihood, complete-data log-likelihood, and expected complete-data likelihood w.r.t. the conditional distribution of missing labels.

Here is a detailed tutorial on how that works in the context of a mixture of regressions: Fitting mixtures of linear regressions.

Hope that helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.