My objective is to implement a topic model for a large number of documents (20M or 30M). Let us assume that the number of topics is fixed at 50.

I think implementing an LDA for the above problem would not be difficult. However, I have yet to find an answer for an NMF model. I have read that it is NOT easy to implement an NMF model for a large number of documents.

Is it really not possible to implement an NMF model for my problem?

  • 5
    $\begingroup$ I suppose NMF=non-negative matrix factorisation and LDA=latent Dirichlet allocation? It is usually a good idea to explain what abbreviations mean. $\endgroup$ – Momo Oct 11 '12 at 11:05
  • $\begingroup$ Thanks a lot, Momo. Any suggestions? $\endgroup$ – Learning_Spark Oct 11 '12 at 17:31
  • $\begingroup$ I don't know the answer, sorry. Maybe try something distributed like research.microsoft.com/pubs/119077/DNMF.pdf $\endgroup$ – Momo Oct 11 '12 at 19:20

Note on implementing LDA for this problem: there are well-designed inference algorithms for huge numbers of documents. Specifically, you should check out "Online LDA", which can adaptively train the topics looking at small chunks of documents at a time.

Paper: http://www.cs.princeton.edu/~blei/papers/HoffmanBleiBach2010b.pdf

Matt Hoffman has python code available: http://www.cs.princeton.edu/~blei/topicmodeling.html


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.