Questions tagged [distance-functions]
Distance functions refer to functions used for quantifying the notion of distance between members of a set, or between objects.
301
questions
0
votes
0answers
10 views
Performance measure for estimation for acoustic impulse response
In searching for a performance measure for assert the estimation quality of acoustic impulse responses.
Ideal Acoustic Impulse Responses (AIRs) are usually modelled as trains of impulses:
$$ h(t) = \...
-1
votes
1answer
12 views
How to unsupervised-cluster of binary vectors?
I have a set of binary vectors of roughly 500 dimensions. For EDA purposes mainly, I'd like to cluster them, maybe hierarchically.
What could be the right distance metric for my problem?
Is the ...
1
vote
0answers
14 views
Determine outliers for robust Mahalanobis distance
I want to apply a robust mahal distance and found an implementation in scikit: https://scikit-learn.org/stable/auto_examples/covariance/plot_mahalanobis_distances.html
but there is the number of ...
1
vote
0answers
28 views
Distance measure for two probability distribution of unequal sample size
Context: I have 100 stores and these stores are divided into 10 business markets. I want to select 3 markets where each market is a good representation of the 100 stores i.e. the population.
There ...
0
votes
0answers
29 views
Understanding the kdist graph used to select DBSCAN epsilon parameter
I need to use DBSCAN for my research and am having trouble understanding the kdist graph used to select the epsilon parameter - specifically, I do not understand what is happening behind the scenes ...
1
vote
0answers
33 views
How to transform $P[k_1\leq (x_i-\mu - \sigma\cdot Z)^2 \leq k_2]$ to $P[k_1\leq \frac{(x_i-\mu)^2}{\sigma^2}+e \leq k_2]$?
Taste estimation
As an example consider an experiment conducted to determine the optimal concentration of salt in popcorn. The concentration of salt in sample $i$ is denoted by ${x_i}$.
The subject ...
1
vote
1answer
22 views
Understanding PAM - why is it greedy?
I've been studying k-medoids for a while but i can't understand the first step or BUILD step: in particular i can't get how the initial medoids would be "greedy". I'm not much confident with the ...
0
votes
0answers
20 views
How to measure the difference between two random forest models?
Suppose that I have training data defined as a set of N records (or samples) defined by its attributes (or descriptors, features, as you prefer), and I trained two random forest models with two parts ...
0
votes
0answers
10 views
Multiple correspondence analysis, definition of distance between two categories of the same question
From the text : Multiple Correspondents Analysis by Brigette LeRoux
The data for this quesiton is:
The definition of $f_k$ is
$f_k = n_k/n$ where $n$ is the total number of individuals and $n_k$ ...
0
votes
1answer
27 views
Mahalanobis distance - understanding the formula [duplicate]
I've read quite a few explanations on this topic, liking this one the most:
https://mccormickml.com/2014/07/22/mahalanobis-distance/
But there is still one thing I don't understand.
I understand ...
0
votes
0answers
19 views
Type of logarithm in Jenson-Shannon and Bhattacharyya distance
Both Jenson-Shannon and Bhattacharyya distance can be used to measure the similarity of two probability distributions.
Bhattacharyya distance between two distributions $p$ and $q$ is defined as $D_B(...
1
vote
1answer
16 views
Dissimilarity or distance metrics between pairs of values
I have cubes (objects) in which their volume was manually calculated (let's call this method the "manual method". Assume that the volume measures obtained by this method are considered the "true" ...
0
votes
0answers
6 views
Similarity matrix from Frechet distance
I have a Frechet Distance matrix and I need to transfer it to the Similarity matrix. Can I use same approach like here ?
1
vote
1answer
66 views
Am I right that Calinski-Harabasz index (Pseudo-F) can not be calculated from a distance matrix other than euclidean?
Part:
I wonder if one could calculate the Calinski-Harabasz index when only having a distance matrix (and a cluster solution, of course). As you need the within and between sum of squares to come up ...
1
vote
0answers
13 views
Does Mahalanabis Distance have something to do with Min-Max normalisation? [duplicate]
Does Mahalanabis Distance have something to do with Min-Max normalisation?
I know that it has something to do with Z-score normalisation, but when I tried Mahalanabis Distance on the Min-max ...
0
votes
0answers
15 views
Calculating a single Factor value against a table of variables
Ok here's a fun one...
We have to calculate a given referee's strength rating/appropriateness for refereeing a given game. The number we come up with has to be something meaningful, so it should be ...
0
votes
1answer
150 views
Is Hierarchical clustering a special case of knn(specific n=1)?
I'am working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
0
votes
0answers
6 views
Dispersion index and Cosine distance
I have different sets of vectors of dimension n and I want to measure and compare the dispersion index of each set.
For my specific problem, each set contains vector representations of words (...
3
votes
1answer
46 views
Non-metrics give “pathological” solutions: what does this mean?
In this set of slides on DTW, slide 25 says that we generally prefer metrics over measures because,
"Non-Metrics can sometimes give pathological solutions when clustering or classifying data etc."
...
3
votes
0answers
94 views
Which distance metric to use to cluster categorical sequences (clickstreams or clickpaths)?
For my research, I want to cluster website visitors based on their clickstreams to understand different information behavior patterns (i.e., customer/visitor journeys). The data can be characterized ...
0
votes
0answers
18 views
How to Measure/Quantify Similarity between > 2 Sets of Dummies Indicators
I have multiple sets, each with a binary indicator for whether an element is part of the set.
...
0
votes
0answers
46 views
Signature of a distribution - Earth Mover distance
I am studying the Earth Mover Distance from here, but I have some difficulty in fully understanding what is the signature of a distribution and how it matches with the last constraint of the Earth ...
0
votes
0answers
75 views
Total variation norm
I am reading
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012, January). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science ...
0
votes
0answers
18 views
Proxy for Mahalanobis distance when n < p? [duplicate]
I'm working on a ranking problem where I want to measure the distance between a collection of query points (as a group) and each target point in my database. Each query point is part of the set of ...
0
votes
0answers
21 views
determine suitable values for the parameters of the distance function for this graph
Hi I've been learning data mining and came across this question. I couldn't seem to figure it out myself.
So we have a large single undirected graph(without attributes) G = (V,E) and want to detect ...
0
votes
0answers
363 views
Jaccard Index for Binary Data in R using dist function
I have a presence/absence table of 0s and 1s and I would like to cluster this data. I want to create a pairwise matrix using R's dist function which has a binary ...
1
vote
1answer
20 views
Finding the effect of nodes on a density heatmap
Let's say I have a geo-tagged dataset of all payment transactions for businesses in a city. I know whether each payment is made by cash or card, and have made a heatmap of where in the city the ...
1
vote
0answers
175 views
Relationship between KL divergence, JS divergence, and MMD?
What kind of relationship is there between the KL (Kullback-Leibler) divergence, JS (Jensen-Shannon) divergence, and MMD (maximum mean discrepancy)?
I know that they all share a global minimum at $P=...
1
vote
1answer
485 views
Is one-hot encoding and standardization of data equivalent to Gower's distance?
For clustering and other techniques for mixed data (numerical and categorical), Gower's distance is usually more preferred than Euclidean distance because the former computes distance differently for ...
0
votes
0answers
37 views
Similarity measure before and after dimensionality reduction or clustering
I have a dataset with 500 000 samples, each sample contains 30 features.
The values of the features are in the range 0.0 to 1.0.
...
0
votes
0answers
18 views
Mahalanobis: Covariance or correlation? [duplicate]
Is the Mahalanobis distance using correlations or covariances (among two vectors) to determine the similarity? I know that both are quite the same and differ only by a division of a standard deviation ...
1
vote
0answers
42 views
Distance between two distributions
I am analyzing time-series data, and I would like to detect a significant change in data distribution. I already know about Bhattacharyya distance, but that requires histograms of equal-sized bins. ...
4
votes
1answer
483 views
Calculation of relative distance
I have 100 term triplets as shown in the below mentioned figure. Each triplet contains 3 objects namely x, y and z. I want to rank the triplets according to the following two properties.
y should be ...
3
votes
1answer
145 views
The violation of triangle inequality in KNN
If the 0<p<1 in the distance metrics, then the triangle inequality is violated.
The question as follows
Does the violation of this inequality affect the ...
1
vote
0answers
30 views
Distance metrics [closed]
I read about different distance metrics, such as Euclidean distance and so on.
What are the machine learning algorithms that use distance metrics in their calculations?
2
votes
1answer
570 views
How can I order kmeans clusters?
I have a kmeans cluster object and I would like to order the clusters. Not the observations within the clusters, rather the clusters in order of each other.
Is there a way of doing this? I found ...
-1
votes
1answer
229 views
Cleaning Up the Data with Mahalanobis Distance [duplicate]
I have a data set and I want to cleaning up my data set from the ouliers, so I decide to use the Mahalanobis distance to find the outliers. But I have a problem here since my covariance matrix isn't ...
1
vote
0answers
170 views
What happens with Mahalanobis-Distance, when the assumption of equal Covariance-Matrices breaks down
Assume that we want to compare the forecast quality of various forecasters $f$ on $n$ values such as stock-market prices or whatever. We could then define a "Mahalanobis-Distance" (MD) (or rather ...
0
votes
0answers
54 views
What distance measure (e.g. Jaccard) to use when there are correlated observations?
I would like to use a simple distance measure (e.g. Jaccard) on a rather sparse set of 0/1 variables or probability densities, for which some of the observations are similar; this is due to repeated ...
0
votes
1answer
314 views
Is Earth Mover Distance has maximum bound?
I have two probability distributions which each distribution has sum up to 1.
I want to compute the distance between those two probability distributions. I want to use Earth Mover Distance to ...
4
votes
2answers
620 views
Is the maximum bound of Euclidean distance between two probability distributions equal to $\sqrt{2}$?
I used Euclidean distance to compute the distance between two probability distribution. The example of computation shown in the Figure below.
As my understanding, the maximum distance occur while $...
2
votes
0answers
82 views
Recovering a distance matrix from nonnegative sparse correlation matrix?
After doing extensive literature research in all sorts of science I am completely puzzled. I am trying to find out what the state-of-the-art techniques would be to recover a (let's say euclidean) ...
2
votes
1answer
157 views
When does the Wasserstein metric attain inequality WLOG?
I’m reading a classic paper [1] that describes a version of the Wasserstein metric (aka Mallows metric), defined as follows. Let $F$ and $G$ be probabilities in $\mathbb{R} ^B$, and let $U \sim F$ and ...
3
votes
2answers
117 views
Distance calculation on variables that cannot be represented in the Euclidean space
Given a variable such as number of events attended together, which is more of a multi-dimensional data how can you calculate a sort of distance between people (i.e. ...
1
vote
0answers
25 views
Name of divergence $D(p||q) = \max_x {p(x) - q(x)}$
$D(p||q) = \max_x {p(x) - q(x)}$ is thought of as a very simple divergence for two continuous probability distributions.
This satisfies minimum requirements of divergence, $D(p||q)\ge0$ and $D(p||q)=...
0
votes
1answer
245 views
K-means clustering on a large matrix using kendall's tau as a distance measure
I'm trying to use kmeans clustering on a relatively large matrix (4000x4000) using the amap::Kmeans function but R seems to be freezed even after more than half an hour. I have to restart R after this....
3
votes
2answers
217 views
Distance metric for source code
I'm trying to compare source code from multiple github projects, and in particular I'm looking for projects that include large chunks of code from other repositories, or large chunks with small ...
0
votes
1answer
35 views
How to compare hundreds of one-dimensional functions
I have hundreds (2002 in today's case) of functions of the distance between two objects: $f_1(x)$, $f_2(x)$, ..., $f_{2002}(x)$. They are correlation functions. They all converge to 0 when $x$ is ...
1
vote
0answers
24 views
How to calculate the similarity/distance between multiple measures for a single individual
I have what may be a relatively simple query, but I'm unsure of the best way to do it. I would like to calculate the similarity between multiple measures for a single individual. I have a data matrix ...
1
vote
2answers
188 views
calculating distance to 1 (positive and negative numbers)
I have what I assume is a very simple question.
I have a range of numbers from 81 to -6, that corresponds to racios pop. growth/urban construction of cities. The ideal ratio would be = 1. I need, ...
