Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My objective is to implement a topic model for a large number of documents (20M or 30M). Let us assume that the number of topics is fixed at 50.

I think implementing an LDA for the above problem would not be difficult. However, I have yet to find an answer for an NMF model. I have read that it is NOT easy to implement an NMF model for a large number of documents.

Is it really not possible to implement an NMF model for my problem?

share|improve this question
I suppose NMF=non-negative matrix factorisation and LDA=latent Dirichlet allocation? It is usually a good idea to explain what abbreviations mean. – Momo Oct 11 '12 at 11:05
Thanks a lot, Momo. Any suggestions? – Learning_Spark Oct 11 '12 at 17:31
I don't know the answer, sorry. Maybe try something distributed like – Momo Oct 11 '12 at 19:20
Thanks a lot, Momo. – Learning_Spark Oct 12 '12 at 10:53

Note on implementing LDA for this problem: there are well-designed inference algorithms for huge numbers of documents. Specifically, you should check out "Online LDA", which can adaptively train the topics looking at small chunks of documents at a time.


Matt Hoffman has python code available:

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.