I am calculating the correlations between vectors of experimental data by a variety of methods (Pearson's, cosine similarity, Euclidian distance, etc.). Most the results look fine, but occasionally the cosine similarity will seem extremely inflated.
For example, these two vectors don't appear to be correlated. The Pearson's correlation coefficient is 0.16 (this makes sense). The cosine similarity is 0.995. Why is it so high?
# two vectors x1 <- c(2000,2000,2000,1987.5,2000,2000,2000,1910,1926.66666666667,2000,1723.33333333333,2000,2000,1992.5,2000,2000,2000,1997.5,2000,1516.66666666667) x2 <- c(10000,8045,10000,9985,10000,10000,10000,8866.66666666667,10000,8285,8173.33333333333,10000,9590,9355,10000,10000,9990,10000,7345,9557.5) # visually, they seem mostly uncorrelated plot(x1, x2) # Pearson's correlation coefficient is 0.16 (this makes sense) cor(x1, x2) # The cosine similarity is 0.995. Why? cossim <- function(A, B) sum(A*B) / ( sqrt(sum(A^2)) * sqrt(sum(B^2)) ) # cosine similarity cossim(x1, x2)
Edit: by request, I added an image that includes the origin in the axes scales. I guess I understand the motivation for this, but I still don't see why this is a 0.995. I would imagine that the vectors are more than 0.005 "different."