My objective is to implement a topic model for a large number of documents (20M or 30M). Let us assume that the number of topics is fixed at 50.

I think implementing an LDA for the above problem would not be difficult. However, I have yet to find an answer for an NMF model. I have read that it is NOT easy to implement an NMF model for a large number of documents.

Is it really not possible to implement an NMF model for my problem?

  • 5
    $\begingroup$ I suppose NMF=non-negative matrix factorisation and LDA=latent Dirichlet allocation? It is usually a good idea to explain what abbreviations mean. $\endgroup$
    – Momo
    Oct 11, 2012 at 11:05
  • $\begingroup$ Thanks a lot, Momo. Any suggestions? $\endgroup$ Oct 11, 2012 at 17:31
  • $\begingroup$ I don't know the answer, sorry. Maybe try something distributed like research.microsoft.com/pubs/119077/DNMF.pdf $\endgroup$
    – Momo
    Oct 11, 2012 at 19:20

1 Answer 1


Note on implementing LDA for this problem: there are well-designed inference algorithms for huge numbers of documents. Specifically, you should check out "Online LDA", which can adaptively train the topics looking at small chunks of documents at a time.

Paper: http://www.cs.princeton.edu/~blei/papers/HoffmanBleiBach2010b.pdf

Matt Hoffman has python code available: http://www.cs.princeton.edu/~blei/topicmodeling.html


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.