2
$\begingroup$

I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only contains 20 documents or so. So How can I calculate the similarity between the two corpora? Any good ideas?

I think the more overlapping words / documents between two corpora, the more similar they are. Currently I have two simple methods implemented, one is

a * DocumentVectorSimilarityOfTwocorpora + (1-a) * WordVectorSimilarityOfTwocorpora,

the other one is to represent each corpus as a document-word matrix, then unfold it into a one way vector, then use cosine similarity.

Both of them are simple ways, I am wondering if there are more accurate methods. Is there a way to obtain the distributions of implicit topics of the two corpora and compute cosine similarity of the two topic distribution vectors of them?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.