My question may be a silly one. So I shall apologize in advance.

I was trying to use the GLOVE model pre-trained by Stanford NLP group (link). However, I noticed that my similarity results showed some negative numbers.

That immediately prompted me to look at the word-vector data file. Apparently, the values in the word vectors were allowed to be negative. That explained why I saw negative cosine similarities.

I am used to the concept of cosine similarity of frequency vectors, whose values are bounded in [0, 1]. I know for a fact that dot product and cosine function can be positive or negative, depending on the angle between vector. But I really have a hard time understanding and interpreting this negative cosine similarity.

For example, if I have a pair of words giving similarity of -0.1, are they less similar than another pair whose similarity is 0.05? How about comparing similarity of -0.9 to 0.8?

Or should I just look at the absolute value of minimal angle difference from $n\pi$? Absolute value of the scores?

Many many thanks.

  • 1
    $\begingroup$ Cosine similarity tag says: An angular-type similarity coefficient between two vectors. It is like correlation, only without centering the vectors. The only difference between the two is that in correlation deviations (moments) - which are being cross-multiplied - are from the mean, while in cosine deviations are from the original 0 - i.e. they are the values as they are. $\endgroup$ – ttnphns Feb 26 '16 at 22:24
  • 2
    $\begingroup$ (cont.) Understanding of positive or negative coefficient is the same in both instances. Negative coef. means that positive deviations/values of one vector tend to pair with negative deviations/values of the other. Whether this means the vectors are "similar" or on the contrary "highly dissimilar" is dependent on what is the meaning of positive and negative deviations/values in the data, for you. $\endgroup$ – ttnphns Feb 26 '16 at 22:28
  • $\begingroup$ @ttnphns Thank you very much for your comment! It does inspire me to think about cosine similarity in a new way. In my use case, perhaps I can think of it as a difference in end results: if the correlation of Doc A and B is negative, and an uni-topical journal X includes Doc A, then it is less likely that X includes B as well, from some mean probability. Does this interpretation sounds valid to you? $\endgroup$ – Mai Feb 26 '16 at 22:40
  • $\begingroup$ I dare not to say because I don't know your data, meaning of values in it, and your study. $\endgroup$ – ttnphns Feb 26 '16 at 22:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.