# Interpreting negative cosine similarity

My question may be a silly one. So I shall apologize in advance.

I was trying to use the GLOVE model pre-trained by Stanford NLP group (link). However, I noticed that my similarity results showed some negative numbers.

That immediately prompted me to look at the word-vector data file. Apparently, the values in the word vectors were allowed to be negative. That explained why I saw negative cosine similarities.

I am used to the concept of cosine similarity of frequency vectors, whose values are bounded in [0, 1]. I know for a fact that dot product and cosine function can be positive or negative, depending on the angle between vector. But I really have a hard time understanding and interpreting this negative cosine similarity.

For example, if I have a pair of words giving similarity of -0.1, are they less similar than another pair whose similarity is 0.05? How about comparing similarity of -0.9 to 0.8?

Or should I just look at the absolute value of minimal angle difference from $n\pi$? Absolute value of the scores?

Many many thanks.

• Cosine similarity tag says: An angular-type similarity coefficient between two vectors. It is like correlation, only without centering the vectors. The only difference between the two is that in correlation deviations (moments) - which are being cross-multiplied - are from the mean, while in cosine deviations are from the original 0 - i.e. they are the values as they are. – ttnphns Feb 26 '16 at 22:24
• (cont.) Understanding of positive or negative coefficient is the same in both instances. Negative coef. means that positive deviations/values of one vector tend to pair with negative deviations/values of the other. Whether this means the vectors are "similar" or on the contrary "highly dissimilar" is dependent on what is the meaning of positive and negative deviations/values in the data, for you. – ttnphns Feb 26 '16 at 22:28
• @ttnphns Thank you very much for your comment! It does inspire me to think about cosine similarity in a new way. In my use case, perhaps I can think of it as a difference in end results: if the correlation of Doc A and B is negative, and an uni-topical journal X includes Doc A, then it is less likely that X includes B as well, from some mean probability. Does this interpretation sounds valid to you? – Mai Feb 26 '16 at 22:40
• I dare not to say because I don't know your data, meaning of values in it, and your study. – ttnphns Feb 26 '16 at 22:55

Do not use the absolute values, as the negative sign is not arbitrary. To acquire a cosine value between 0 and 1, you should use the following cosine function:

(R code)

cos.sim <- function(a,b)
{
dot_product = sum(a*b)
anorm = sqrt(sum((a)^2))
bnorm = sqrt(sum((b)^2))
minx =-1
maxx = 1
return(((dot_product/anorm*bnorm)-minx)/(maxx-minx))
}


(Python Code)

def cos_sim(a, b):
"""Takes 2 vectors a, b and returns the cosine similarity according
to the definition of the dot product"""
dot_product = np.dot(a, b)
norm_a = np.linalg.norm(a)
norm_b = np.linalg.norm(b)
return dot_product / (norm_a * norm_b)

minx = -1
maxx = 1

cos_sim(row1, row2)- minx)/(maxx-minx)
$$$$
`
• Where do you set minx and maxx? You might apply this min-max normalization to vector dimensions, instead of calculated distance. – emre can Nov 26 '19 at 2:34

Cosine similarity is just like Pearson correlation, but without substracting the means. So you can compare the relative strengh of 2 cosine similarities by looking at the absolute values, just like how you would compare the absolute values of 2 Pearson correlations.