# Fast alternatives to the EM algorithm

Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.

-
Have you done a literature survey? This paper looks particularly relevant: Convex Relaxations of Latent Variable Training –  Emre Apr 24 '12 at 8:54
How about LSA? :-) –  conjugateprior Apr 24 '12 at 12:21
A general way to accelerate an EM is called "Aitken accelerator". If precision is not an issue, maybe try moment estimation or generalized moment estimation instead. –  JohnRos Apr 29 '12 at 19:42

Newton-Raphson algorithms can often be employed. I am not familiar with pSLA, but it is pretty common to use Newton-Raphson algorithms for latent class models. Newton-Raphson algorithms are a little more troubled by poor initial values than EM, so one strategy is to first use a few iterations (say 20) of the EM and then switch to a Newton-Raphson algorithm. One algorithm that I have had a lot of success with is: Zhu, Ciyou, Richard H. Byrd, Peihuang Lu, and Jorge Nocedal (1997), "Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization," ACM Transactions on Mathematical Software (TOMS) archive, 23 (4), 550-60.

-