My question may be a silly one. So I shall apologize in advance.

I was trying to use the GLOVE model pre-trained by Stanford NLP group (link). However, I noticed that my similarity results showed some negative numbers.

That immediately prompted me to look at the word-vector data file. Apparently, the values in the word vectors were allowed to be negative. That explained why I saw negative cosine similarities.

I am used to the concept of cosine similarity of frequency vectors, whose values are bounded in [0, 1]. I know for a fact that dot product and cosine function can be positive or negative, depending on the angle between vector. But I really have a hard time understanding and interpreting this negative cosine similarity.

For example, if I have a pair of words giving similarity of -0.1, are they less similar than another pair whose similarity is 0.05? How about comparing similarity of -0.9 to 0.8?

Or should I just look at the absolute value of minimal angle difference from $n\pi$? Absolute value of the scores?

Many many thanks.

  • 1
    $\begingroup$ Cosine similarity tag says: An angular-type similarity coefficient between two vectors. It is like correlation, only without centering the vectors. The only difference between the two is that in correlation deviations (moments) - which are being cross-multiplied - are from the mean, while in cosine deviations are from the original 0 - i.e. they are the values as they are. $\endgroup$ – ttnphns Feb 26 '16 at 22:24
  • 2
    $\begingroup$ (cont.) Understanding of positive or negative coefficient is the same in both instances. Negative coef. means that positive deviations/values of one vector tend to pair with negative deviations/values of the other. Whether this means the vectors are "similar" or on the contrary "highly dissimilar" is dependent on what is the meaning of positive and negative deviations/values in the data, for you. $\endgroup$ – ttnphns Feb 26 '16 at 22:28
  • $\begingroup$ @ttnphns Thank you very much for your comment! It does inspire me to think about cosine similarity in a new way. In my use case, perhaps I can think of it as a difference in end results: if the correlation of Doc A and B is negative, and an uni-topical journal X includes Doc A, then it is less likely that X includes B as well, from some mean probability. Does this interpretation sounds valid to you? $\endgroup$ – Mai Feb 26 '16 at 22:40
  • $\begingroup$ I dare not to say because I don't know your data, meaning of values in it, and your study. $\endgroup$ – ttnphns Feb 26 '16 at 22:55

Do not use the absolute values, as the negative sign is not arbitrary. To acquire a cosine value between 0 and 1, you should use the following cosine function:

(R code)

cos.sim <- function(a,b) 
  dot_product = sum(a*b)
  anorm = sqrt(sum((a)^2))
  bnorm = sqrt(sum((b)^2))
  minx =-1
  maxx = 1

(Python Code)

def cos_sim(a, b):
    """Takes 2 vectors a, b and returns the cosine similarity according 
to the definition of the dot product"""
    dot_product = np.dot(a, b)
    norm_a = np.linalg.norm(a)
    norm_b = np.linalg.norm(b)
    return dot_product / (norm_a * norm_b)

minx = -1 
maxx = 1

cos_sim(row1, row2)- minx)/(maxx-minx)
  • $\begingroup$ Where do you set minx and maxx? You might apply this min-max normalization to vector dimensions, instead of calculated distance. $\endgroup$ – emre can Nov 26 '19 at 2:34

Cosine similarity is just like Pearson correlation, but without substracting the means. So you can compare the relative strengh of 2 cosine similarities by looking at the absolute values, just like how you would compare the absolute values of 2 Pearson correlations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.