I have two corpora for example, each of which contains a set of different documents, and each document are already represented as a vector of words in a certain way. The two corpora are small, only contains 20 documents or so. So How can I calculate the similarity between the two corpora? Any good ideas?
I think the more overlapping words / documents between two corpora, the more similar they are. Currently I have two simple methods implemented, one is
a * DocumentVectorSimilarityOfTwocorpora + (1-a) * WordVectorSimilarityOfTwocorpora,
the other one is to represent each corpus as a document-word matrix, then unfold it into a one way vector, then use cosine similarity.
Both of them are simple ways, I am wondering if there are more accurate methods. Is there a way to obtain the distributions of implicit topics of the two corpora and compute cosine similarity of the two topic distribution vectors of them?