1
$\begingroup$

em algorithm is usually optimized iteratively between the expectation (construct a lower-bound) and the maximize the likelihood (optimize the lower-bound) to guarantee convergence. However, at each iteration of the algorithm we must use the whole data to maximize the likelihood. Now I want to use the em algorithm with a single sample or a mini-batch at each iteration. So how to optimize in the expectation step and re-estimate parameters in the maximization step?

[See here]https://math.stackexchange.com/questions/2958733/how-can-i-do-online-expectation-maximization-algorithm

$\endgroup$
  • 1
    $\begingroup$ What type of data are you dealing with that requires EM? $\endgroup$ – Xiaomi Oct 27 '18 at 2:26
  • $\begingroup$ The data I want to deal with is the continuous feature data. It is about several hundreds dimension for each data item. However, our data is coming one by one instead of be collected in advance. So I must update the model with online algorithm. $\endgroup$ – yuanyuan Oct 27 '18 at 5:11
  • $\begingroup$ If you aren't limited by tight execution time constraints, you can use an offline, i.e., batch, algorithm instead of an online algorithm. You can use the optimal parameter values from the previous problem, i.e., before the most recent data point was added, as starting (initial) point for the optimization solver.. After you have several data points, re-running the optimization with one additional data point might be very fast if the optimal values for the problem with the added point are close to the optimal values without that added point. $\endgroup$ – Mark L. Stone Oct 27 '18 at 12:34
  • $\begingroup$ Due to the optimal values for the problem with the added point are close to the optimal values without the added point, It's true that re-running the optimization with one additional data point will have a faster convergence after I have several data points. However, simply re-running means that the running time for adding a data point will continue to increase. Because that as time goes, the number of data points will increase and it will spend more time to run a single update. So I hope the solution is to obtain the new optimal value only with the current model and the added points. $\endgroup$ – yuanyuan Oct 28 '18 at 2:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.