Questions tagged [distance-functions]

Distance functions refer to functions used for quantifying the notion of distance between members of a set, or between objects.

Filter by
Sorted by
Tagged with
0
votes
1answer
27 views

High Dimensional Swiss Roll? (For Metric Learning/Dimensionality Reduction)

So I've just started a project which includes some metric learning, and came accross this swiss roll in 3D to 2D problem. Ideally, you should 'unroll' the roll. My question is, can this be extended ...
0
votes
0answers
12 views

Splitting method for time series

Say I have a website with a lot of pages and traffic. Some of them are more visited, some of them a less. My task is to split pages into two groups for the given time period, so that the total traffic ...
1
vote
0answers
27 views

How to Show Wasserstein Metric is Sum Invariant?

A paper I'm trying to understand states that that Wasserstein metric obeys certain properties, which I'd like to prove. This metric is defined for two random variables $U, V$ and $p \in (1, \infty)$ ...
0
votes
0answers
33 views

Measuring the amount of total influence between variables?

I have a group of people with a series of yes/no questuins about preferences in various topics. The thing is they could see each other answers and change theirs hence might be influenced by a common ...
2
votes
1answer
40 views

Do $k$-means, dbscan, and hierarchical clustering all rely on (pseudo)metrics?

I seems to me that the clustering methods $k$-means, dbscan, and hierarchical clustering all work on distance measures $d$ that are (pseudo)metrics, i.e., fulfill the following requirements: $$ d(x,x)=...
2
votes
2answers
110 views

Is Jaccard similarity/distance suitable for non-binary, quantitative data?

I have a dataset with each row a country and 10 columns with numerical features like GDP,Electrcity consumption, GNI etc. I am trying to use distance metrics to find similarity between the countries ...
0
votes
0answers
19 views

Creating a covariance matrix from a distance matrix—normalizing matrix with infinity

I'm trying to simulate spatially autocorrelated data by creating a covariance matrix from a distance matrix and using said covariance matrix as the $\Sigma$ parameter of a multivariate normal. I have ...
0
votes
0answers
13 views

How to calculate similarity between two sets of items rated on a single dimension?

(I'm just making up variables for this example.) Let's say I have 100 words rated on their pleasantness. I also have 100 images rated on their pleasantness. I then had participants rate the fit ...
0
votes
1answer
48 views

Is there a distance metric between ordered vectors?

I am looking for a distance metric between vectors whose elements are ordered, i.e so the vectors: [1,5,0,0,0,0,1], [1,0,4,0,0,1,0] will be considered closer than [1,5,0,0,0,0,1], [1,0,0,1,5,0,0] for ...
0
votes
0answers
12 views

Metric to measure distance between two percentage values [duplicate]

I would like to find a function that measure how much two percentage values differ from each other. Now, a norm $\|\cdot \|$ does not quite fit this criterion as a jump from 1% to 30% is, for many ...
0
votes
1answer
25 views

Ordering list of items by two criteria

I have a list of items with two scores: scoreA and scoreB. To be more specific they represent the average of a list of accuracy scores and their maximum. Both of the scores range from 0 to 100%. I'm ...
0
votes
0answers
36 views

Jaccard/binary (dis)similarity calculation to multidimensional scaling analysis

I have n N x n dataset of features (n) and subjects (N) in which I am attempting to cluster into a lower-dimensional space via multi-dimensional scaling. I'm confused about which MDS setup I need to ...
0
votes
0answers
15 views

Distance metric for probability distribution

I'm looking for a distance metric that would be good for what I think is a probability distribution: I have a fixed number of samples I have a fixed number of features Each sample can have, at most,...
0
votes
0answers
6 views

How to define a weight function for response time?

I have a repeated measurement over time and the variable of interest is user compliance. However, since users use different questionnaire length, the duration of response time is also important. I ...
0
votes
0answers
6 views

Distance in Square between Randomly Selected [duplicate]

I am trying to find the expected Euclidean distance between independent, randomly-selected variables in the unit square and I have some technical questions. For co text, I know if we were selecting ...
0
votes
0answers
25 views

What is the difference between np.linalg.norm(x-y,axis=1) and np.linalg.norm(x-y)?

I'm creating a K-Medoids algorithm from scratch in Python using numpy, and I'm in the process of using a distance function to determine the cluster center. I want the center to be the point in the ...
0
votes
0answers
12 views

distance between two lists

I am using the normalized manhattan distance (L1-norm) between two lists as a metric to measure how much change has happened. Let's call this changeIdx .This ...
2
votes
0answers
19 views

Similarity between clusters/groups?

I have a dataset consisting of multiple groups in a high dimensional space. An example is shown below: What would be the best way to calculate similarities between groups. Say how similar is group A ...
1
vote
0answers
102 views

How to calculate Mahalanobis distance in very high dimensions with both continuous and categorical variables?

The objective is outlier detection via a distance measure. Does mahalanobis distance suffer from curse of dimensionality just like Eucledian distance for very high dimensions? (say around 5000 ...
0
votes
0answers
59 views

Finding weights to be used by an Euclidean distance function for vectors with weighted components in a multi-dimensional space, using only sample data

I have a multi-dimensional space with $d$ dimensions wherein $i$ vectors ($v_1 ... v_i$) with $d$ components live. I want to find a function $s(v_a, v_b)$ which takes in two of these vectors and ...
2
votes
0answers
35 views

Hierarchical clustering: distance/linkage combination that allows starting in the middle of the dendrogram

I want to use hierarchical clustering to classify some ecological data (species abundances on different places), so I would like to use a Manhattan type distance that doesn't account for double ...
0
votes
0answers
10 views

Finding the relative 'importance' of different vector components when defining distance of two vectors in a space [duplicate]

I have a multi-dimensional space with $d$ dimensions wherein $i$ vectors ($v_1 ... v_i$) with $d$ components live. I want to find a function $s(v_a, v_b)$ which takes in two of these vectors and ...
1
vote
1answer
38 views

Metric for distance of sinewaves

I have time-series data consisting of the sum of 2 sinewaves and my goal is to predict their frequencies and their amplitudes. I would like to know what are the best distance metrics/loss functions I ...
0
votes
0answers
10 views

Performance measure for estimation for acoustic impulse response

In searching for a performance measure for assert the estimation quality of acoustic impulse responses. Ideal Acoustic Impulse Responses (AIRs) are usually modelled as trains of impulses: $$ h(t) = \...
-1
votes
1answer
32 views

How to unsupervised-cluster of binary vectors?

I have a set of binary vectors of roughly 500 dimensions. For EDA purposes mainly, I'd like to cluster them, maybe hierarchically. What could be the right distance metric for my problem? Is the ...
2
votes
0answers
22 views

Determine outliers for robust Mahalanobis distance

I want to apply a robust mahal distance and found an implementation in scikit: https://scikit-learn.org/stable/auto_examples/covariance/plot_mahalanobis_distances.html but there is the number of ...
1
vote
0answers
54 views

Distance measure for two probability distribution of unequal sample size

Context: I have 100 stores and these stores are divided into 10 business markets. I want to select 3 markets where each market is a good representation of the 100 stores i.e. the population. There ...
0
votes
0answers
163 views

Understanding the kdist graph used to select DBSCAN epsilon parameter

I need to use DBSCAN for my research and am having trouble understanding the kdist graph used to select the epsilon parameter - specifically, I do not understand what is happening behind the scenes ...
1
vote
0answers
34 views

How to transform $P[k_1\leq (x_i-\mu - \sigma\cdot Z)^2 \leq k_2]$ to $P[k_1\leq \frac{(x_i-\mu)^2}{\sigma^2}+e \leq k_2]$?

Taste estimation As an example consider an experiment conducted to determine the optimal concentration of salt in popcorn. The concentration of salt in sample $i$ is denoted by ${x_i}$. The subject ...
1
vote
1answer
40 views

Understanding PAM - why is it greedy?

I've been studying k-medoids for a while but i can't understand the first step or BUILD step: in particular i can't get how the initial medoids would be "greedy". I'm not much confident with the ...
0
votes
0answers
43 views

How to measure the difference between two random forest models?

Suppose that I have training data defined as a set of N records (or samples) defined by its attributes (or descriptors, features, as you prefer), and I trained two random forest models with two parts ...
0
votes
1answer
58 views

Mahalanobis distance - understanding the formula [duplicate]

I've read quite a few explanations on this topic, liking this one the most: https://mccormickml.com/2014/07/22/mahalanobis-distance/ But there is still one thing I don't understand. I understand ...
0
votes
0answers
35 views

Type of logarithm in Jenson-Shannon and Bhattacharyya distance

Both Jenson-Shannon and Bhattacharyya distance can be used to measure the similarity of two probability distributions. Bhattacharyya distance between two distributions $p$ and $q$ is defined as $D_B(...
1
vote
1answer
19 views

Dissimilarity or distance metrics between pairs of values

I have cubes (objects) in which their volume was manually calculated (let's call this method the "manual method". Assume that the volume measures obtained by this method are considered the "true" ...
1
vote
1answer
218 views

Am I right that Calinski-Harabasz index (Pseudo-F) can not be calculated from a distance matrix other than euclidean?

Part: I wonder if one could calculate the Calinski-Harabasz index when only having a distance matrix (and a cluster solution, of course). As you need the within and between sum of squares to come up ...
1
vote
0answers
14 views

Does Mahalanabis Distance have something to do with Min-Max normalisation? [duplicate]

Does Mahalanabis Distance have something to do with Min-Max normalisation? I know that it has something to do with Z-score normalisation, but when I tried Mahalanabis Distance on the Min-max ...
1
vote
1answer
440 views

Is Hierarchical clustering a special case of knn(specific n=1)?

I'm working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
3
votes
1answer
57 views

Non-metrics give “pathological” solutions: what does this mean?

In this set of slides on DTW, slide 25 says that we generally prefer metrics over measures because, "Non-Metrics can sometimes give pathological solutions when clustering or classifying data etc." ...
3
votes
0answers
133 views

Which distance metric to use to cluster categorical sequences (clickstreams or clickpaths)?

For my research, I want to cluster website visitors based on their clickstreams to understand different information behavior patterns (i.e., customer/visitor journeys). The data can be characterized ...
0
votes
0answers
30 views

Proxy for Mahalanobis distance when n < p? [duplicate]

I'm working on a ranking problem where I want to measure the distance between a collection of query points (as a group) and each target point in my database. Each query point is part of the set of ...
1
vote
1answer
23 views

Finding the effect of nodes on a density heatmap

Let's say I have a geo-tagged dataset of all payment transactions for businesses in a city. I know whether each payment is made by cash or card, and have made a heatmap of where in the city the ...
2
votes
0answers
297 views

Relationship between KL divergence, JS divergence, and MMD?

What kind of relationship is there between the KL (Kullback-Leibler) divergence, JS (Jensen-Shannon) divergence, and MMD (maximum mean discrepancy)? I know that they all share a global minimum at $P=...
2
votes
1answer
1k views

Is one-hot encoding and standardization of data equivalent to Gower's distance?

For clustering and other techniques for mixed data (numerical and categorical), Gower's distance is usually more preferred than Euclidean distance because the former computes distance differently for ...
1
vote
0answers
48 views

Distance between two distributions

I am analyzing time-series data, and I would like to detect a significant change in data distribution. I already know about Bhattacharyya distance, but that requires histograms of equal-sized bins. ...
4
votes
1answer
1k views

Calculation of relative distance

I have 100 term triplets as shown in the below mentioned figure. Each triplet contains 3 objects namely x, y and z. I want to rank the triplets according to the following two properties. y should be ...
3
votes
1answer
201 views

The violation of triangle inequality in KNN

If the 0<p<1 in the distance metrics, then the triangle inequality is violated. The question as follows Does the violation of this inequality affect the ...
1
vote
0answers
31 views

Distance metrics [closed]

I read about different distance metrics, such as Euclidean distance and so on. What are the machine learning algorithms that use distance metrics in their calculations?
3
votes
1answer
1k views

How can I order kmeans clusters?

I have a kmeans cluster object and I would like to order the clusters. Not the observations within the clusters, rather the clusters in order of each other. Is there a way of doing this? I found ...
-1
votes
1answer
307 views

Cleaning Up the Data with Mahalanobis Distance [duplicate]

I have a data set and I want to cleaning up my data set from the ouliers, so I decide to use the Mahalanobis distance to find the outliers. But I have a problem here since my covariance matrix isn't ...
1
vote
0answers
213 views

What happens with Mahalanobis-Distance, when the assumption of equal Covariance-Matrices breaks down

Assume that we want to compare the forecast quality of various forecasters $f$ on $n$ values such as stock-market prices or whatever. We could then define a "Mahalanobis-Distance" (MD) (or rather ...

1
2 3 4 5
7