0
$\begingroup$

I am currently implementing a word2vec model that uses the cosine similarity to determine the similarity between two vectors. When plotting all the possible cosine similarities, I get the following graph:

enter image description here

The graph is pretty skewed towards the 'most similar value'. However, I am not sure if this is an ok thing, which can just be corrected by normalizing the data, or if this skewness indicates that something is terribly wrong with the model.

I am not an expert on the topic of natural language processing. Can you guys provide me with some intuition what if it is ok to normalize, or if this indicates if the model is terribly wrong? Any ideas and insights are appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

In word2vec or doc2vec implementations most values of cosine similarity are between -0.2 and 0.2. All vectors are similar with a few number of other vectors and dissimilar with the most. This is more plausible than the inverse, that is your skewness towards most similar values. Most vectors cannot be similar, because this would say that in a text there is not evident semantic difference between words.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.