What is the difference between pairwise kernels and pairwise distances?

Question

What is the difference between pairwise kernels and pairwise distances?

I frequently came across terms like pairwise kernels and pairwise distances while learning about Pairwise metrics, Affinities, and Kernels, but I don't get the real meaning of these words and their differences.

Answer 1

I think to start; we should understand concerning Kernel(-functions):

• How to intuitively explain what a kernel is?
• Is a kernel function just a mapping?
• What is a kernel function?

Pairwise distance provides distance between two vectors/arrays. So the more pairwise distance, the less similarity while cosine similarity is: $cosine\_similarity = (1 - pairwise\_distance)$, so the more cosine similarity, the more similarity between two vectors/arrays.

Please see the following example to find Cosine similarity performance on two random quotes/strings using TF-IDF:

from sklearn.feature_extraction.text import TfidfVectorizer

documents = (
    "The Worst Distance Between Two People is Misunderstanding",
    "A Mutual Misunderstanding"    
)

tfidf_vectorizer = TfidfVectorizer()
tfidf_matrix = tfidf_vectorizer.fit_transform(documents)

#method 1:
from sklearn.metrics.pairwise import cosine_similarity
print(cosine_similarity(tfidf_matrix[0:1], tfidf_matrix)[0,1])
#0.1505569696020495

#method 2:
from sklearn.metrics.pairwise import pairwise_distances
print(pairwise_distances(tfidf_matrix[0:1], tfidf_matrix, metric='cosine')[0,1])
#0.8494430303979505

Note 1: that according to documentation: Cosine distance is defined as 1.0 minus the cosine similarity.

$$Cosine\_distance = (1 - cos(similarity))$$

That explains the difference between the two methods is used for similarity computation for quotes.

Note 2: based on scikit-learn documentation valid metrics for Pairwise distance are :

[‘cityblock’, ‘cosine’, ‘euclidean’, ‘l1’, ‘l2’, ‘manhattan’]. These metrics support sparse matrix inputs. [‘nan_euclidean’] but it does not yet support sparse matrices.

From scipy.spatial.distance documentation, valid metrics for Pairwise distance matrix are:

[‘braycurtis’, ‘canberra’, ‘chebyshev’, ‘correlation’, ‘dice’, ‘hamming’, ‘jaccard’, ‘kulsinski’, ‘mahalanobis’, ‘minkowski’, ‘rogerstanimoto’, ‘russellrao’, ‘seuclidean’, ‘sokalmichener’, ‘sokalsneath’, ‘sqeuclidean’, ‘yule’]

Note 3: According to this answer, definitions for cosine distance that they used are different in scipy & sklearn. The one used in sklearn is a measure of similarity while the one used in scipy is a measure of dissimilarity

Concerning Pairwise distance measures, which many ML-based algorithms (supervised\unsupervised) use the following distance measures/metrics:

1. Euclidean Distance
2. Cosine Similarity
3. Hamming Distance
4. Manhattan Distance
5. Chebyshev Distance
6. Minkowski
7. Jaccard Index
8. Haversine
9. Sørensen-Dice Index
10. Gower’s Distance

There is also interesting visualization from sklearn here.

This article by M. Grootendorst gives you better intuition to understand the choices of distance metrics as well as the advantages and pitfalls of 9 frequently used distance measures in Data Science.

In the case of categorical data, you must convert it to numeric data, either by ranking (for an ordered factor) or by encoding it as a set of binary (dummy) variables. If the data consists of mixed continuous and binary variables, you will usually want to scale the variables so that the ranges are similar; one way is to use Gower’s distance. Selecting the proper distance measure resulted in a much better model!