0
$\begingroup$

Why is the top result obtained using cosine similarity extremely close to 0 not the expected 1?

That implies complete orthogonality.

Data: 100k documents/rows with 2000 features(TF_IDF values of tokens, mostly zeroes/sparse)

Used NearestNearest neighbors with cosine similarity from this thread:

How to find nearest neighbors using cosine similarity for all items from a large embeddings matrix?

# import NearestNeighbors
from sklearn.neighbors import NearestNeighbors
n_neighbor = 10

model = NearestNeighbors(n_neighbors=n_neighbor,
                         metric='cosine',
                         algorithm='brute',
                         n_jobs=-1)
model.fit(df_tfidf)
# let's check first 3 documents
model.kneighbors(df_tfidf.iloc[:3, :])

Out:
(array([[2.22044605e-16, 5.96041399e-01, 6.02695589e-01, 6.05811405e-01,
         6.06062605e-01, 6.06394780e-01, 6.10842747e-01, 6.12047848e-01,
         6.12241848e-01, 6.13224358e-01],
        [0.00000000e+00, 4.26139518e-01, 4.29424721e-01, 4.29424721e-01,
         4.29424721e-01, 4.35292677e-01, 4.41962536e-01, 4.51103908e-01,
         4.52376335e-01, 4.54416818e-01],
        [2.22044605e-16, 6.13395642e-01, 6.20716271e-01, 6.38328419e-01,
     6.43845160e-01, 6.60639675e-01, 6.61006505e-01, 6.66453839e-01,
     6.70172442e-01, 6.71674788e-01]]),
 array([[     0,  40197,   7517,  41572,  40659,  95641,  15538,  36000,
           3115,  36789],
        [     1, 117889,  78484,  37726,  72102,  75169,  91649,   4410,
          18514,  88808],
        [     2,  80311,  63605,  40658,  18017,  33410,  14809,   8880,
           2964,    157]], dtype=int64))

So we can see that the document is similar to itself - index matches. The cosine similarity for this top match is 0 not the expected 1.

Thus the vector appears completely transposed

Are the other 9 matches trustworthy - that is are they not similarly transposed?

Notably they are sorted in ascending order.

So the results come out my top match, then 10th match, 9th match, ... 2nd match

It seems like an inconsistency in the model displaying results.

Surely you would want the identical document to have a score of 1 not 0?

EDIT2: I rerun the model looking for 50 closest matches and got the exact same results for the first 10 matches and then 40 more increasing values.

So are the cosine similarity values here inverted - that is 1-real value?

EDIT PER Community Bot request: My question is why kneighbors gives similarity score of 0 for the top match not the expected 1 in the case of document itself.

$\endgroup$
1
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Commented Feb 27, 2023 at 11:08

1 Answer 1

1
$\begingroup$

My hypothesis that model.kneighbors returns 1 - real_cos_similarity score appears correct.

I took a small 10 document x 2000 tf_idf score matrix from the original data and computed cosine similarity using

# import cosine_similarity
from sklearn.metrics.pairwise import cosine_similarity

# compute cosine similarity
cosine_sim = cosine_similarity(df_tfidf.iloc[:10, :])
cos_rounded = cosine_sim.round(2)


# convert kneighbors 
one_minus = (1 - kneighbors[0]).round(2)

print(one_minus[0])
# [1.  , 0.2 , 0.15, 0.08, 0.08, 0.06, 0.06, 0.05, 0.03, 0.02]

The results from one_minus makes sense now they are in sorted descending sequence now of similarity scores as expected.

print(cos_rounded[0])
# [1.  , 0.03, 0.08, 0.05, 0.15, 0.08, 0.02, 0.06, 0.06, 0.2 ]

Results from cos_rounded are as expected the scores are not sorted, first row has perfect match with itself.

# compare set of values in first row of one_minus with first row of  
cos_rounded

set(one_minus[0]) ==  set(cos_rounded[0])

Out: True

Only question is why does it happen that kneighbors returns 1 - real_cos_similarity?

$\endgroup$
1
  • $\begingroup$ As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Commented Feb 28, 2023 at 11:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.