2
$\begingroup$

I've been experimenting with LDA topic modelling using Gensim. I couldn't seem to find any topic model evaluation facility in Gensim, which could report on the perplexity of a topic model on held-out evaluation texts thus facilitates subsequent fine tuning of LDA parameters (e.g. number of topics). It would be greatly appreciated if anyone could shed some light on how I can perform topic model evaluation in Gensim. This question had also been posted on Stackoverflow.

$\endgroup$
2
$\begingroup$

Found the answer on the gensim mailing list.

In short, the bound() method of LdaModel computes a lower bound on perplexity, based on a held-out corpus.

$\endgroup$
2
$\begingroup$

This seemingly recent (and great) addition to GitHub by Markus Konrad seems to address MANY Topic Modeling evaluation problems:

tmtoolkit - Text Mining and Topic modeling tool kit:

https://github.com/WZBSocialScienceCenter/tmtoolkit

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.