15
$\begingroup$

Inspired by this question, I'm wondering whether any work has been done on topic models for large collections of extremely short texts. My intuition is that Twitter should be a natural inspiration for such models. However, from some limited experimentation, it looks like standard topic models (LDA, etc) perform quite poorly on this kind of data.

Does anyone out there know of any work which has been done in this area? This paper talks about applying LDA to Twitter, but I'm really interested in whether there are other algorithms which perform better in the short-document context.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Twitter is a particularly difficult dataset for topic modeling not only due to the small size of the 'documents', but also due to the type of text. People tend to use various texting shorthands which makes identifying co-occurrences even more difficult. $\endgroup$
    – Nick
    Mar 30, 2012 at 22:57
  • $\begingroup$ See the list of good papers and the corresponding source codes for topic modeling on Tweets at: quora.com/… $\endgroup$
    – NQD
    Mar 15, 2017 at 15:15

3 Answers 3

Reset to default
8
$\begingroup$

This is a late answer, but it can be useful for other people searching for related research and tools for this problem:

  1. Weiwei Guo from Columbia implemented code for short-text topic modeling. He described the implementation in the paper "Modeling Sentences in the Latent Space" (http://aclweb.org/anthology-new/P/P12/P12-1091v2.pdf) and the code is available here: http://www.cs.columbia.edu/~weiwei/code.html

  2. Although this is not topic modeling, if you have a classification task involving short pieces of texts, you can use LibShortText. From their web site description

"LibShortText is an open source tool for short-text classification and analysis. It can handle the classification of, for example, titles, questions, sentences, and short messages..."

http://www.csie.ntu.edu.tw/~cjlin/libshorttext/

Improve this answer
$\endgroup$
7
$\begingroup$

While I'm not super familiar with his work, I know Jacob Eisenstein has done work in text analysis and graphical models in twitter data. In particular, this paper describes an application of topic modeling in twitter data and microblogs.

Edit: actually after reading the paper a bit more, they state:

However, the average message on Twitter is only sixteen word tokens, which is too sparse for traditional topic modeling; instead, we gathered together all of the messages from a given user into a single document.

So perhaps that very paper may not be of much help, still maybe other Eisenstein publications may lead you in the right direction.

Improve this answer
$\endgroup$
7
$\begingroup$

A recent paper called "a biterm topic model for short text" (WWW13) has made some progress on this topic, and here is its code

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ I confirm that BiTerm LDA worked pretty well for short text utterances (3-8 words) topic modeling and subsequent classification. $\endgroup$
    – Vladislavs Dovgalecs
    Jan 26, 2016 at 8:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.