I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only contains 20 documents or so. So How can I calculate the similarity between the two corpora? Any good ideas?

I think the more overlapping words / documents between two corpora, the more similar they are. Currently I have two simple methods implemented, one is

a * DocumentVectorSimilarityOfTwocorpora + (1-a) * WordVectorSimilarityOfTwocorpora,

the other one is to represent each corpus as a document-word matrix, then unfold it into a one way vector, then use cosine similarity.

Both of them are simple ways, I am wondering if there are more accurate methods. Is there a way to obtain the distributions of implicit topics of the two corpora and compute cosine similarity of the two topic distribution vectors of them?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.