5
$\begingroup$

I am interested in finding clusters of words / topics in text. I am trying to learn more about potential approaches. The Wikipedia page on document clustering seems to provide a helpful overview (although I am sure there are limitations to this article and would be more than happy to be pointed to other resources).

In a research project, I've been using a two-step clustering approach (hierarchical and then k-means) with some degree of success but am interested in better understanding the landscape.

The Wikipedia article referenced above distinguishes between 'hard' and 'soft' approaches. According to it, hard approaches, such as hierarchical approaches and k-means, assign documents to a single cluster, while soft clustering approaches (the article says that both Latent Drichlet Allocation and topic models are examples of this approach) assign a mixture of clusters to a document. As a caveat, I'm not sure how Latent Drichlet Allocation and topic models can be considered clustering approaches (or whether they are), but my question is, what are the benefits to 'hard' and 'soft' approaches?

In short:

  • What are the benefits of using hierarchical and / or k-means clustering algorithms to identify clusters of words / topics in text?
  • What are the benefits of using Latent Drichlet Allocation / topic models to identify clusters of words / topics in text?
Improve this question
$\endgroup$
2
  • $\begingroup$ What do you mean with "clusters of words in text"? It sounds more like identifying frequent n-grams than topic modelling. $\endgroup$
    – user79309
    Aug 24, 2015 at 12:36
  • $\begingroup$ You're right. I meant something more along the lines of "clusters of documents" and the most frequent words that appear in those documents, with the overall goal of grouping documents on the basis of their similarity and trying to determine why they are similar. $\endgroup$
    – Joshua Rosenberg
    Aug 24, 2015 at 12:39

1 Answer 1

Reset to default
3
$\begingroup$

A classical clustering algorithm (like k-means or hierarchical clustering) gives you one label per document.

Topic modeling gives you a probabilistic composition of the document (so a document has a set of weighted labels). In addition, it gives you topics that are probability distributions over words.

Note that both procedures are unsupervised learning and far from being perfect, no matter how impressing the results may look at first sight. Apply them to dataset you understand well first!

Improve this answer
$\endgroup$
3
  • $\begingroup$ I see. What if a clustering algorithm is combined with something like a cosine metric to compare a document to those in each cluster (e.g., something like Latent Semantic Analysis)? $\endgroup$
    – Joshua Rosenberg
    Aug 24, 2015 at 12:56
  • $\begingroup$ @Joshua Rosenberg Most clustering algorithms require some kind of metric space, otherwise you cannot use them. Usually one uses Euclidean distance (aka cosine metric) for that. $\endgroup$
    – user79309
    Aug 24, 2015 at 13:03
  • $\begingroup$ Got it. What about when a metric space is used to compare one document to the mean word frequencies for the documents in a cluster? $\endgroup$
    – Joshua Rosenberg
    Aug 24, 2015 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.