I have a corpus of publications in CS divided by year.

What I'd like to discover from it is

  • The subject (only one) of each article ( for example testing, software engineering, networking, architecture, etc)

  • Does an article not fall in any subject (so is it a new subject, at least compared to the subjects of the previous year)?

Right now I'm experimenting with LDA, but

  1. The topic it finds are not easily mapped with the ones I'd like to have (for example some topics are general, like about writing papers, or performing experiments. Some other are really hard to understand what they are about. In general they will not be as clearly defined as the ones is stated before)

  2. I need to provide the number of topics. I'd like to discover if there was a change in topic from one year to the other, but as long as I provide the number of topics LDA will always find that number topics

  3. Each document belongs to multiple topics, while I'd like to give just one label (even if that means losing information and doing some kind of approximation when there is a paper which might fall in 2 or more labels)

Of course these are drawbacks which are known and expected from the algorithm, but I'm not sure what alternatives there are to make up for them.

I can not label the collection (what I'm trying to do is exactly labeling the collection in an automatic way), but I can provide papers (which are not in the collection) about a known subject.

What are the algorithms best suited for this kind of task? I'm considering LDA (which I'm testing now, in addition to a clustering-LDA algorithm), LSA and LSI.

How can the latter 2 score compared to LDA? I'd need algorithms which are already available in some kind of library/framework (like gensim).

To try and label my documents I'm thinking about training LDA over my corpus, then transforming documents on a specific subject with the trained LDA, and looking for the documents in my corpus which better match that distribution (for example I transform 100 papers from about testing and look for papers in my corpus with a topic distribution similar to those). Could this method work (or will it produce rubbish)? Does exist a (sound) algorithm that does something similar to that?

  • I doubt any unsupervised algorithm will ever be able to do this. The algorithm would need a general understanding of the language and the domain. – Anony-Mousse Dec 2 '15 at 7:24
  • @Anony-Mousse What could be a supervised alternative? Does an algorithm exist which I can train the way I specified in the question? I'm not working with a big corpus, it's around 3500 documents for a total of 65 MB of text (the one I'm interested in classifying), but I can provide many more labeled documents. – Makers_F Dec 2 '15 at 7:38
  • You have to use supervised approaches, if you want anything "human-like". – Anony-Mousse Dec 2 '15 at 8:42
  • @Anony-Mousse Could you suggest a supervised approach and possibly a framework which implements it? Python would be preferable, if possible – Makers_F Dec 2 '15 at 9:01
  • 1
    I'm not sure what the best topic-learning approach is, not my domain. – Anony-Mousse Dec 2 '15 at 13:55

Supervised latent Dirichlet allocation seems the most natural fit. In this model, the representation of each document is augmented with some sort of response variable $Y$ (see illustration from the paper below). (In your case, a categorical variable denoting subject, or a one-hot vector equivalent.)

Illustrated model for supervised latent Dirichlet allocation

Because both the observed words and the response variable depend on the topics, this will produce different topic distributions from standard LDA. The paper's experiments demonstrated that this was more effective for prediction than standard LDA followed by regression:

We compared sLDA to linear regression on the $\bar\varphi_d $ from unsupervised LDA. This is the regression equivalent of using LDA topics as classification features [4]. Figure 2 (L) illustrates that sLDA provides improved predictions on both data sets. Moreover, this improvement does not come at the cost of document model quality. The per-word hold-out likelihood comparison in Figure 2 (R) shows that sLDA fits the document data as well or better than LDA.

Of course, this requires labelled documents.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.