What are some of the Document Clustering approaches with high recall? I am looking for document clustering approaches which gives high recall. I tried looking at Google but all I get is TF-IDF and K-means. Are there more sophisticated approaches than that which achieve a high recall?
Thanks!
Edit as per suggestions:
I have tried employing LDA using gensim. The results are rather bad. I removed the stop words, did stemming and also removed all punctuation. However when I try to infer a topic in the test, the most probable topic of each document is almost the same (>98% of the times). I have tried with 50, 100, 200, 300 and 400 topics, all give same results.
Attached is the distribution of 200(orange) topics and 300(blue) topics. (Sorry about the wrong title.)


Also, attached is the visualization of topics by pyLDAVis for 200 topics.

 A: The problem with TF-IDF for clustering is that you will be working in a very high dimensional space. There's  a phenomenon called the curse of dimensionality whereby methods that work in low dimensions fall apart in higher dimensions. This is particularly true of high-dimensional clustering like you find in text analysis.
In general, you have a couple choices (as outlined in the wiki article). A great primer for your case is Chapter 4 of a textbook by Charu Aggarwal (thankfully provided free by the author here) that covers text clustering.
I've used Latent Dirichlet Allocation (LDA) to reduce the dimensionality to topics (as opposed to words). Then you can apply K-means to the "topic vectors", which will often be much lower dimensional than the vocabulary space.
An addition, you should consider using a fractional distance metric as opposed to the standard Euclidean Metric. Aggarwal and his collaborators published a nice paper on this here. In many cases, the blind use of the $L_2$ norm will lead to poor classification accuracy.
