What method for grouping documents by topic I have a corpus of publications in CS divided by year.
What I'd like to discover from it is


*

*The subject (only one) of each article ( for example testing, software engineering, networking, architecture, etc)

*Does an article not fall in any subject (so is it a new subject, at least compared to the subjects of the previous year)?
Right now I'm experimenting with LDA, but


*

*The topic it finds are not easily mapped with the ones I'd like to have (for example some topics are general, like about writing papers, or performing experiments. Some other are really hard to understand what they are about. In general they will not be as clearly defined as the ones is stated before)

*I need to provide the number of topics. I'd like to discover if there was a change in topic from one year to the other, but as long as I provide the number of topics LDA will always find that number topics

*Each document belongs to multiple topics, while I'd like to give just one label (even if that means losing information and doing some kind of approximation when there is a paper which might fall in 2 or more labels)
Of course these are drawbacks which are known and expected from the algorithm, but I'm not sure what alternatives there are to make up for them.
I can not label the collection (what I'm trying to do is exactly labeling the collection in an automatic way), but I can provide papers (which are not in the collection) about a known subject.
What are the algorithms best suited for this kind of task?
I'm considering LDA (which I'm testing now, in addition to a clustering-LDA algorithm), LSA and LSI.
How can the latter 2 score compared to LDA?
I'd need algorithms which are already available in some kind of library/framework (like gensim).
To try and label my documents I'm thinking about training LDA over my corpus, then transforming documents on a specific subject with the trained LDA, and looking for the documents in my corpus which better match that distribution (for example I transform 100 papers from about testing and look for papers in my corpus with a topic distribution similar to those).
Could this method work (or will it produce rubbish)? Does exist a (sound) algorithm that does something similar to that?
 A: Supervised latent Dirichlet allocation seems the most natural fit. In this model, the representation of each document is augmented with some sort of response variable $Y$ (see illustration from the paper below). (In your case, a categorical variable denoting subject, or a one-hot vector equivalent.) 

Because both the observed words and the response variable depend on the topics, this will produce different topic distributions from standard LDA. The paper's experiments demonstrated that this was more effective for prediction than standard LDA followed by regression:

We compared sLDA to linear regression on the $\bar\varphi_d $ from unsupervised LDA. This is the regression equivalent of using LDA topics as classification features [4]. Figure 2 (L) illustrates that sLDA provides
  improved predictions on both data sets. Moreover, this improvement does not come at the cost of document model quality. The per-word hold-out likelihood comparison in Figure 2 (R) shows that
  sLDA fits the document data as well or better than LDA.

Of course, this requires labelled documents.
