I have a corpus of publications in CS divided by year.
What I'd like to discover from it is
The subject (only one) of each article ( for example testing, software engineering, networking, architecture, etc)
Does an article not fall in any subject (so is it a new subject, at least compared to the subjects of the previous year)?
Right now I'm experimenting with LDA, but
The topic it finds are not easily mapped with the ones I'd like to have (for example some topics are general, like about writing papers, or performing experiments. Some other are really hard to understand what they are about. In general they will not be as clearly defined as the ones is stated before)
I need to provide the number of topics. I'd like to discover if there was a change in topic from one year to the other, but as long as I provide the number of topics LDA will always find that number topics
Each document belongs to multiple topics, while I'd like to give just one label (even if that means losing information and doing some kind of approximation when there is a paper which might fall in 2 or more labels)
Of course these are drawbacks which are known and expected from the algorithm, but I'm not sure what alternatives there are to make up for them.
I can not label the collection (what I'm trying to do is exactly labeling the collection in an automatic way), but I can provide papers (which are not in the collection) about a known subject.
What are the algorithms best suited for this kind of task? I'm considering LDA (which I'm testing now, in addition to a clustering-LDA algorithm), LSA and LSI.
How can the latter 2 score compared to LDA? I'd need algorithms which are already available in some kind of library/framework (like gensim).
To try and label my documents I'm thinking about training LDA over my corpus, then transforming documents on a specific subject with the trained LDA, and looking for the documents in my corpus which better match that distribution (for example I transform 100 papers from about testing and look for papers in my corpus with a topic distribution similar to those). Could this method work (or will it produce rubbish)? Does exist a (sound) algorithm that does something similar to that?