4
$\begingroup$

What is the difference between pairwise kernels and pairwise distances?

I frequently came across terms like pairwise kernels and pairwise distances while learning about Pairwise metrics, Affinities, and Kernels, but I don't get the real meaning of these words and their differences.

$\endgroup$

1 Answer 1

2
$\begingroup$

I think to start; we should understand concerning Kernel(-functions):

Pairwise distance provides distance between two vectors/arrays. So the more pairwise distance, the less similarity while cosine similarity is: $cosine\_similarity = (1 - pairwise\_distance)$, so the more cosine similarity, the more similarity between two vectors/arrays.

Please see the following example to find Cosine similarity performance on two random quotes/strings using TF-IDF: img

from sklearn.feature_extraction.text import TfidfVectorizer

documents = (
    "The Worst Distance Between Two People is Misunderstanding",
    "A Mutual Misunderstanding"    
)

tfidf_vectorizer = TfidfVectorizer()
tfidf_matrix = tfidf_vectorizer.fit_transform(documents)

#method 1:
from sklearn.metrics.pairwise import cosine_similarity
print(cosine_similarity(tfidf_matrix[0:1], tfidf_matrix)[0,1])
#0.1505569696020495

#method 2:
from sklearn.metrics.pairwise import pairwise_distances
print(pairwise_distances(tfidf_matrix[0:1], tfidf_matrix, metric='cosine')[0,1])
#0.8494430303979505

Note 1: that according to documentation: Cosine distance is defined as 1.0 minus the cosine similarity.

$$Cosine\_distance = (1 - cos(similarity))$$

That explains the difference between the two methods is used for similarity computation for quotes.

Note 2: based on scikit-learn documentation valid metrics for Pairwise distance are :

[‘cityblock’, ‘cosine’, ‘euclidean’, ‘l1’, ‘l2’, ‘manhattan’]. These metrics support sparse matrix inputs. [‘nan_euclidean’] but it does not yet support sparse matrices.

From scipy.spatial.distance documentation, valid metrics for Pairwise distance matrix are:

[‘braycurtis’, ‘canberra’, ‘chebyshev’, ‘correlation’, ‘dice’, ‘hamming’, ‘jaccard’, ‘kulsinski’, ‘mahalanobis’, ‘minkowski’, ‘rogerstanimoto’, ‘russellrao’, ‘seuclidean’, ‘sokalmichener’, ‘sokalsneath’, ‘sqeuclidean’, ‘yule’]

Note 3: According to this answer, definitions for cosine distance that they used are different in scipy & sklearn. The one used in sklearn is a measure of similarity while the one used in scipy is a measure of dissimilarity

Concerning Pairwise distance measures, which many ML-based algorithms (supervised\unsupervised) use the following distance measures/metrics:

  1. Euclidean Distance
  2. Cosine Similarity
  3. Hamming Distance
  4. Manhattan Distance
  5. Chebyshev Distance
  6. Minkowski
  7. Jaccard Index
  8. Haversine
  9. Sørensen-Dice Index
  10. Gower’s Distance

img There is also interesting visualization from sklearn here.

This article by M. Grootendorst gives you better intuition to understand the choices of distance metrics as well as the advantages and pitfalls of 9 frequently used distance measures in Data Science.

In the case of categorical data, you must convert it to numeric data, either by ranking (for an ordered factor) or by encoding it as a set of binary (dummy) variables. If the data consists of mixed continuous and binary variables, you will usually want to scale the variables so that the ranges are similar; one way is to use Gower’s distance. Selecting the proper distance measure resulted in a much better model!

$\endgroup$
2
  • $\begingroup$ This doesn't seem to address the original question, which was about the difference between distances and kernel functions $\endgroup$
    – user20160
    Nov 24, 2021 at 0:24
  • $\begingroup$ @user20160 The title of the question is a bit vague. I assumed that OP is interested in the context of distance metrics between pairwise kernels and pairwise distances as the link in question discusses this; otherwise, the definition/difference of kernel/pairwise distance does not make any sense! If you have a better explanation for OP, feel free to edit my answer. The understanding of kernel has been addressed at the beginning of the answer and followed by an example to address how distance could be measured pairwisely and finally ended up with some discussion and some references. $\endgroup$
    – Mario
    Nov 24, 2021 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.