Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
Tell me more
×
Cross Validated is a question and answer site for
statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
|
|
|||||||||||||
|
|
Newton-Raphson algorithms can often be employed. I am not familiar with pSLA, but it is pretty common to use Newton-Raphson algorithms for latent class models. Newton-Raphson algorithms are a little more troubled by poor initial values than EM, so one strategy is to first use a few iterations (say 20) of the EM and then switch to a Newton-Raphson algorithm. One algorithm that I have had a lot of success with is: Zhu, Ciyou, Richard H. Byrd, Peihuang Lu, and Jorge Nocedal (1997), "Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization," ACM Transactions on Mathematical Software (TOMS) archive, 23 (4), 550-60. |
|||
|
|
|
Very similar to the EM algorithm is the MM algorithm which typically exploits convexity rather than missing data in majorizing or minorizing an objective function. You have to check if MM algorithm is applicable for your particular problem, though. |
|||
|
|
|
For LDA, "online LDA" is fast alternative to than batch methods like standard EM (http://www.cs.princeton.edu/~blei/papers/HoffmanBleiBach2010b.pdf). David Blei provides software on his page: http://www.cs.princeton.edu/~blei/topicmodeling.html |
|||
|
|
