0
$\begingroup$

I start off with word2vec embeddings and process them in the following way:

  1. Standardize dimensions to mean 0 and standard deviation of 1
  2. PCA to keep the top k-dimensional eigenvector, whereby whitening is active
  3. Normalize eigenvectors to unit length as described in https://stackoverflow.com/questions/41387000/cosine-similarity-of-word2vec-more-than-1

When I then calculate cosine distances between the resulting vectors I still get values greater than 1, which shouldn't be the case?

So I am wondering if the normalization to unit length that I am doing in step 3 makes sense at all, after having done the previous steps? If it does make sense, do you have any ideas what could be the cause for getting cosine distance values greater than 1?

I am using sklearn for all steps.

$\endgroup$

1 Answer 1

1
$\begingroup$

I think I have solved it. Since the vectors produced in step 3 are defined R space rather than R positive space, cosine similarity is defined within [1,-1]. Therefore cosine distance is defined within [0,2]. When I divide my obtained values by 2 I get values ranging between 0 and 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.