0
$\begingroup$

I used Latent Semantic Analysis (LSA) to extract latent topics (i.e., the polynomials coefficient1*word1 + coefficient2*word2 + ...) from a certain corpus. I know that the larger the (absolute value of the) coefficient, the more important the word for that topic. But is there a better metric? Python's gensim package uses this metric, but I couldn't find any textbook or paper that discusses the issue. Any pointers will be greatly appreciated!

$\endgroup$
2
$\begingroup$

Note that the dimensions created by LSA are not all equal. The 1st dimension captures more variation than the second, and so on. So words that map to high (or very negative) values in the first topic are more salient that those that do so on the second topic, and so on (in that they capture more variation in the data). You can test tis by projecting single word documents into the LSA space. Note that the absolute size determines how important the word is, and the sign indicates polarity of the topic, so look at words by abs(topic value).

LSA really learns more representative columns over the data, that are not always topics, per se. If you want topic analysis, LDA may be a better bet. LSA is most commonly used to conflate related words into the same columns in lower dimensional space, to conflate synonyms and related words together to produce better similarity judgements between documents. You can also try word2vec and do some analysis on the learned word vectors.

However, we may be over-complicating things. Typically the tf.idf value of a word in a document is a good indicator of how important it is at describing that document: http://en.wikipedia.org/wiki/Tf%E2%80%93idf gensim has modules for computing that. You can look at word idf scores alone over the whole corpus to get an idea of saliency, ignoring very low frequency words.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.