Suppose we have several documents. These docs are classified into several categories. But there could be issues like these:

  • The categories may not be properly defined.
  • Or The categories are properly defined, but the docs are not properly classified.

Given the category definitions and some classified docs, I want to tell how well the situation is.

I have tried to use the probability distribution of words to describe the current status of each category. And calculate the dissimilarity of different distributions through Kullback-Leibler Divergence. But I went into some issue here: Calculate the Kullback-Leibler Divergence in practice?

I am wondering if the above way is proper. Is there a canonical way to do it?

(Pardon me if the tags I assigned are not proper.)


1 Answer 1


You could create (normalised) word vectors for each category (using the number of times the word occurs in that category as the weight). Then calculating the cosine similarity between these category vectors.

  • 1
    $\begingroup$ On one side, representing the categories is more properly done using the centroid of them, that is, the averaged vector across all the vectors for the document. In fact, the Rocchio algorithm does so for relevance feedback in Information Retrieval. On the the other side, you may think of using multi-document summarization for representing each category, then using the cosine similarity. $\endgroup$ May 20, 2014 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.