Benefits of clustering algorithms and Latent Dirichlet Allocation / topic models for finding clusters of words / topics in text

Question

I am interested in finding clusters of words / topics in text. I am trying to learn more about potential approaches. The Wikipedia page on document clustering seems to provide a helpful overview (although I am sure there are limitations to this article and would be more than happy to be pointed to other resources).

In a research project, I've been using a two-step clustering approach (hierarchical and then k-means) with some degree of success but am interested in better understanding the landscape.

The Wikipedia article referenced above distinguishes between 'hard' and 'soft' approaches. According to it, hard approaches, such as hierarchical approaches and k-means, assign documents to a single cluster, while soft clustering approaches (the article says that both Latent Drichlet Allocation and topic models are examples of this approach) assign a mixture of clusters to a document. As a caveat, I'm not sure how Latent Drichlet Allocation and topic models can be considered clustering approaches (or whether they are), but my question is, what are the benefits to 'hard' and 'soft' approaches?

In short:

• What are the benefits of using hierarchical and / or k-means clustering algorithms to identify clusters of words / topics in text?
• What are the benefits of using Latent Drichlet Allocation / topic models to identify clusters of words / topics in text?

Answer 1

A classical clustering algorithm (like k-means or hierarchical clustering) gives you one label per document.

Topic modeling gives you a probabilistic composition of the document (so a document has a set of weighted labels). In addition, it gives you topics that are probability distributions over words.

Note that both procedures are unsupervised learning and far from being perfect, no matter how impressing the results may look at first sight. Apply them to dataset you understand well first!