I have a large corpus of over 1.2 million text documents. After applying LDA to a small sample of the corpus, I have determined the number of topics be 34 using topic coherence score. Here, some topics dominate over other topics. Now, when I take a different sample, the topic distribution is pretty much the same. In the sense, the same topics dominate over other topics.

My question is, how to show that this is not by random chance. Or to say that the whole text corpus is made up of mainly these topics.

I am new to statistics and I am not sure if this is the correct way to formulate this problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.