GOAL Graph represantation of "similarity" between key phrases based on documents that mention it in their text, in form of simple graph.

I'm working on vectors, of different length, each value in it as a pair {keyWord,strength}. Every document has such a vector created. Therefore it's not a problem to put it into a 2 dimensional array, and get similar vectors, however for each key-phrase, with fields {document,strengthOfPhraseInIt}.

My question is, what methods can be used to compute a value of "similarity" between such phrase vectors, which represent nodes in my future graph?

What I have thought of:

  • Similarity must be calculated and based on intersection of documents. If word "mountains" has common documents with word "goat", then there should be a similarity.

  • Similarity is also based on the value key inside each vector value. Bound between words having document A common, however lets say, value 4 and 20 respective, is weaker than bond with document B with values 6 and 7 for first and second phrase.

  • Normalisation seems to be a must for the graph to be readable.

What methods could be used here? I'm not familiar with statistics, therefore please show me what possibiliteis exist, and what solutions are known for such problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.