6
$\begingroup$

I am thinking about whether one needs to normalize or weight a topic model by document length (page length)?

I am estimating a topic model using social science (JSTOR) articles, where they vary in length between min 5 pages to max 200 pages. I want to analyse a specific topic, namely, the degree of economic topics in social science articles.

I can see that a similar question was raised back in 2011, but no clear suggestion was reached, as far as I can interpret the discussion:

https://lists.cs.princeton.edu/pipermail/topic-models/2011-February/001171.html

My intuition about this question is somewhat split here.

On the one hand it seems logical that one needs to weight by document length since larger documents (a 200 page long document) will have more pages to refer to a specific topic (in my case “economic) than shorter document (a 5 pages long documents). This will be reflected for example in a document-term-matrix where economic terms (e.g. markets, business, and industry) will have much higher frequency in row for the 200 page document compared to the row of the 5 page document. Moreover, the 200 page document will affect the overall term distribution of words. In other words, the terms of the 200-page document will dominate the term per document ratio for each and every term in the document-term-matrix.

On the other hand, the topic-term ratio seems to adjust for the fact that we have longer and shorter documents in the sample. Even if the term-document ratio is high for longer documents and lower for shorter documents, the relative frequency (proportion of various terms for the longer documents is comparable with the shorter. For example, the shorter document might have a sum of 10 for the frequency (tokens) of economic topics of a total of 30 tokens: gives an economic topic probability of 10/30. Whereas the longer document might have a sum of 100 for the frequency of economic topics of a total of 3000 tokens (all topics): a ratio of 100/3000.

Accordingly, even if the shorter document has fewer tokens for economic topics than the longer document, it is still estimated to be more economic than the longer document.

I am not sure what to conclude from this: can I trust a page-unadjusted LDA results? I am using package topicmodels in R.

Many thanks in advance for your input

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I haven't used topic models much, but I can say that if you are to apply usual clustering methods to un-normalized document-term matrices (even when the dimensionality of the data is reduced with LSA), you'll see that longer articles will tent to cluster together, just because they have more words.

So you may take a look at some of your topics and see if documents inside make sense. Also, try to calculate average length of document per topic and see if the phenomenon I mention takes place or not.

Then you can repeat the same on the unit-normalized data and see if the results make more sense or not.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the input Alexey. Interesting, so what you are suggestion will happen is that longer documents will tend to fall into similar topics just because they are longer. I will check if this is happening. I wonder what people with experience using topic models say about this phenomenon? $\endgroup$
    – Adel
    Commented Jun 11, 2015 at 13:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.