0
$\begingroup$

Let us say I have a topic probability per document, for example:

Doc1: Science:0.6, History:0.2, Politics:0.1, Sports:0.1
Doc2: Science:0.3, History:0.5, Politics:0.1, Sports:0.1
Doc3: Science:0.8, History:0.1, Politics:0.05, Sports:0.05
Doc4: Science:0.2, History:0.2, Politics:0.4, Sports:0.2

It is fairly clear that Doc1 is similar to Doc3. If I use topic modeling as a dimensionality reduction tool, can I use clustering methods to cluster documents on topic space? What would be the appropriate clustering technique and distance function? Will K-means with Euclidean distance suffice? (It seems weird to calculate eucledian distance between two probability distribution)

Also, topics might be correlated, what would be a way to handle that?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Yes, Euclidian distances have been used, see here for an example and further reference. $\endgroup$
    – Ben
    Commented Mar 16, 2014 at 5:06
  • $\begingroup$ For measuring the distance between 2 distributions, you could try Jensen-Shannon distance (which is the square root of Jensen-Shannon divergence). $\endgroup$
    – jlund3
    Commented Apr 22, 2014 at 18:04

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.