Questions tagged [distance-functions]

Distance functions refer to functions used for quantifying the notion of distance between members of a set, or between objects.

103 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8k views

Properties of Levenshtein, N-Gram, cosine and Jaccard distance coefficients - in sentence matching

Let's say I have two strings: string A: 'I went to the cafeteria and bought a sandwich.' string B: 'I heard the cafeteria is serving roast-beef sandwiches today'. ...
• 499
983 views

Compute Shannon entropy between every row of a large, sparse matrix

I have a sparse, binary matrix of user (rows) and items (columns). Each element of this matrix is either 0 or 1: ...
• 22.4k
223 views

Comparing term-frequency distributions with unequal sample sizes?

Background I have several datasets of word frequencies where some datasets have much more data than others: from 3000 samples to 20000 samples. I also have large reference corpora with millions of ...
• 279
1k views

Bhattacharyya distance for three histograms

There is a paper “Auto White Balance Based on the Similarity of Chromaticity Histograms” mention about automatic white balance. One of the key point of this algorithm is how to measure the similarity ...
1k views

Scaling a distance to account for missing values

We can compute the Euclidean distance between two vectors $\mathbf{x}$ and $\mathbf{y}$ by: $$d(\mathbf{x}, \mathbf{y}) = \sqrt{(x_1-y_1)^2 + \ldots + (x_n - y_n)^2}$$ When there are missing values ...
• 944
191 views

Approximating a Poisson distribution using a partially observed Gaussian

Note that the problem specification has changed since the original posting. Thanks to whuber for helping me better specify the question: in an attempt to make the question general, I had left out ...
53 views

Metrics for assessing the quality of prior distributions

Clarification: My purpose is to compare different methods for selecting/creating priors (or perhaps I should refer to them as predictive distributions for a quantity of interest/parameter). I do not ...
• 23.4k
242 views

Which distance metric to use to cluster categorical sequences (clickstreams or clickpaths)?

For my research, I want to cluster website visitors based on their clickstreams to understand different information behavior patterns (i.e., customer/visitor journeys). The data can be characterized ...
• 51
1k views

Why the chi-square distance gives a better in high dimensional space

I am a beginner in machine learning. I did a classification program using KNN using two similarity distances: Euclidean distance and chi-square distance.The size of each feature vector is 10000. I ...
112 views

Name of an $f$-divergence

The term divergence means a function $D$, which, given two probability distributions $P,Q$, assigns a non-negative real number $D(P,Q)$ such that $D(P,Q) = 0$ iff $P(x)=Q(x) \forall x$. The relative ...
• 1,021
740 views

What is a good similarity measure to use when missing data is a significant issue?

I have a list of cities that I want to compare in terms of their similarity. Each city can described by a large but finite number of characteristics but most of them will have missing data for some ...
835 views

What is a good technique for grouping objects based on binary or dichotomous traits?

I have a set of objects each of which has a list of traits. Data on the traits is binary: an object has a trait or does not. The number of objects that I have is moderately greater than the number ...
• 2,617
159 views

I'm searching for a similarity metric such that given two Facebook users it returns a value that reflects how similar the two users are. The similarity metrics must take into account (at the same time)...
• 215
193 views

Similarity Amongst Recipes Using Ingredients and Reviews/Descriptions

I'm still toying with things and just learning this, so please forgive any incorrect terminology. My toy data set is a collection of recipes with a fairly significant overlap in ingredients. I'm ...
• 131
441 views

Similarity measure for weighted sets (multisets)

I have to compute a similarity measure between different sets (Actually they are more like maps than sets). A weight is associated to each element of the set. The sets I want to compare represent ...
176 views

A best measure for speaker recognition

I have a set $E_{1}$, with a finite cardinality $n$ of rectangular matrices which contains the useful MFCC coefficients generated from $n$ speech signals. Similary I have a set $E_{2}$ of same ...
• 71
9k views

Jaccard distance vs Levenshtein distance for fuzzy matching

My data is similar to the following data, but far bigger and more complex. Apple Banana Those fruits Tomato Cocumber These vegetables I would like to get the ...
• 4,963
81 views

How to Show Wasserstein Metric is Sum Invariant?

A paper I'm trying to understand states that that Wasserstein metric obeys certain properties, which I'd like to prove. This metric is defined for two random variables $U, V$ and $p \in (1, \infty)$ ...
130 views

Similarity between clusters/groups?

I have a dataset consisting of multiple groups in a high dimensional space. An example is shown below: What would be the best way to calculate similarities between groups. Say how similar is group A ...
361 views

How to calculate Mahalanobis distance in very high dimensions with both continuous and categorical variables?

The objective is outlier detection via a distance measure. Does mahalanobis distance suffer from curse of dimensionality just like Eucledian distance for very high dimensions? (say around 5000 ...
• 303
71 views

Hierarchical clustering: distance/linkage combination that allows starting in the middle of the dendrogram

I want to use hierarchical clustering to classify some ecological data (species abundances on different places), so I would like to use a Manhattan type distance that doesn't account for double ...
58 views

Determine outliers for robust Mahalanobis distance

I want to apply a robust mahal distance and found an implementation in scikit: https://scikit-learn.org/stable/auto_examples/covariance/plot_mahalanobis_distances.html but there is the number of ...
• 2,309
239 views

Distance measure for two probability distribution of unequal sample size

Context: I have 100 stores and these stores are divided into 10 business markets. I want to select 3 markets where each market is a good representation of the 100 stores i.e. the population. There ...
• 43
534 views

• 225
1 vote
33 views

How to calculate the similarity/distance between multiple measures for a single individual

I have what may be a relatively simple query, but I'm unsure of the best way to do it. I would like to calculate the similarity between multiple measures for a single individual. I have a data matrix ...
• 11
1 vote
1k views

Distance Matrix for Big Data

I have been struggling to create a distance matrix for some Big Data (800,000x20). I have tried R (dist function), Matlab (pdist function), and cloud computing (to increase RAM). Ultimately, the ...
• 86
1 vote