Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

269 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
10
votes
0answers
4k views

How to compare two distance matrices?

Suppose that I have two distance matrices for the same set of items. By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
6
votes
1answer
2k views

How is pairwise PERMANOVA/adonis a valid non-parametric approach for pairwise comparisons

Assume that we have taken independent random samples of several individuals from 5 locations that represent 5 populations. The design is fairly unbalanced: the number of individuals sampled from each ...
6
votes
2answers
538 views

Expected squared distance from origin of training points vs. test points

This is from Exercise 2.4 (Page 39) of Elements of Statistical Learning: The edge effect problem discussed on page 23 is not peculiar to uniform sampling from bounded domains. Consider inputs drawn ...
6
votes
0answers
395 views

What, if any, dissimilarity is preserved in partial least squares (PLS)?

When we perform a principal components analysis (PCA) on a multivariate data set we are interested in finding orthogonal components that explain maximal variance in the data set. We can form a biplot ...
5
votes
0answers
920 views

Measure of similarity between two multivariate samples

I have two samples from a multivariate distribution. One is approximate and the other is exact. My aim is to measure the similarity between the approximate sample and the exact sample. In particular, ...
5
votes
1answer
1k views

Bhattacharyya distance for three histograms

There is a paper “Auto White Balance Based on the Similarity of Chromaticity Histograms” mention about automatic white balance. One of the key point of this algorithm is how to measure the similarity ...
5
votes
0answers
189 views

Joint distribution of two distances

Suppose there are three points in 3D space, each with coordinates $A_i=(X_i,Y_i,Z_i)\leadsto \mathcal{N}(\mu_i,\tau^2\mathbb{I}_3)$. We compute the distance between the three points, e.g. $D_{ij} = \|...
5
votes
0answers
621 views

AIC with Mantel's tests

Mantel's tests are commonly used to compare genetic distances (say, between a number of individuals) with true or hypothesized landscape distances between those same individuals. For example, “does ...
5
votes
1answer
265 views

Determine rate of change in dissimilarity (distance)?

I have repeated measures plant abundance data for 36 forest plots, across 80 years involving 50+ species of trees. The data are structured as: Columns = ...
4
votes
0answers
33 views

Distribution $f$ that minimizes $JSD(f||q) + JSD(f||p)$

What can we say about the distribution $f^*$ that is the solution to the following optimization problem: $$\min_f JSD(f||p)+JSD(f||q) ,$$ where $p,q$ are given distributions over some set, and $JSD$ ...
4
votes
1answer
172 views

Is random walk a good metric to compute distances of two sets of nodes in the graph?

I have a big graph and I have three sets(A,B,C) of labeled nodes on it. I would like to compute on average how fare each sets are from each others. In other word, I want to compute distance matrix for ...
4
votes
0answers
1k views

Scaling a distance to account for missing values

We can compute the Euclidean distance between two vectors $\mathbf{x}$ and $\mathbf{y}$ by: $$ d(\mathbf{x}, \mathbf{y}) = \sqrt{(x_1-y_1)^2 + \ldots + (x_n - y_n)^2} $$ When there are missing values ...
4
votes
0answers
46 views

Sampling vector so they will have a given euclidean distances matrix

Given a matrix $M\in\mathbb{R}^{P\times P}$ , is it possible to sample $P$ vectors $u_i\in\mathbb{R}^N$, $i=1..P$ so that $\|u_i-u_j\|=M_{ij}$. Obviously for not any $M$ this is possible, i.e. it has ...
4
votes
0answers
308 views

Distance between independent observations of a categorical variable

I have a random variable $T$ that takes values in $\{ \text{blue}, \text{green}, \text{red} \}$, and a number of observations of $T$: ...
4
votes
0answers
814 views

Why it is better to use the cumulative distribution to compute distances?

In the comments of this question, it was pointed out that, when comparing two distributions, it is more natural and more general use the cumulative distribution (CDF) instead of the distribution (PDF)....
4
votes
0answers
504 views

Distribution of the hyperbolic distance between random points in the Poincaré disc

Let two points at polar coordinates $(r_i, \theta_i)$ and $(r_j, \theta_j)$ be hyperbolic points in $\mathbb{H}^2_\zeta$ with curvature $K=-\zeta^2$. The radial coordinates of these points are ...
4
votes
0answers
100 views

Is there any way to define a distance metric given a Hidden Markov Model?

Let's say I've gotten a HMM that describes user search strings for my e-commerce website. Let's also say that I've just received a search string from a customer that doesn't have any search results. ...
3
votes
0answers
84 views

Finding a sub-population from dataset matching another target dataset

Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
3
votes
0answers
60 views

What is the appropriate metric for determining distance / dissimilarity of sparse, high dimensional data in PCA space?

I'm working with scRNA-seq data (~96% sparse, high dimensional), and am trying to determine distances between the cells in PCA space - NOT for the specific purpose of clustering. The principal ...
3
votes
0answers
72 views

Are there distance-based models that can include a random effect?

Distance-based linear models, such as those implemented by the adonis package in R, allow us to fit one or more predictors to a multivariate response represented by ...
3
votes
1answer
58 views

Computing Wassertein Distance

For two probability measures $\mu$ and $\nu$, the Wassertein Distance is defined as $$W_p (\mu , \nu) = \left[ \inf\limits_{\gamma \in \Gamma} |x-y|^p \, d\gamma (x,y) \right] ^{\frac{1}{p}} \, , $$ ...
3
votes
0answers
370 views

Distance or divergence for ordinal distribution

Measures like KL divergence can be symmetrized (into JS divergence). Bhattacharyya distance serves a similar function. Either is well-suited to both continuous distributions and discrete (e.g. ...
3
votes
0answers
703 views

Clustering a set of curves

I am working with a MRI dataset where we inject dye into a person's wrist and measure intensity per time on a voxel-by-voxel basis. I am trying to determine if it is possible to identify certain ...
3
votes
0answers
277 views

Is the permutation test p-value valid for a distance metric?

I have two variables, $X \in \mathbb{R}^n$ and $Y \in \mathbb{R}^n$. Let $d$ be a distance metric that satisfies the conditions to be such a metric (non-negativity, symmetry, triangle inequality and ...
3
votes
0answers
75 views

Does the distance metric used with K-Medoids needs to respect the triangle inequality?

I'd like to understand if the K-Medoids algorithm requires to be used with a distance metric that respects the triangle inequality. In particular I'd like to try to apply the K-Medoids algorithm with ...
3
votes
0answers
605 views

Nearest/farthest neighbour between-group distance: an efficient way to find it

This question might be better suited for StackOverflow as it is programming (so you are free to suggest to move it), but it is about a data analysis programming task. The Q: do you know any "elegant" ...
3
votes
0answers
303 views

Should I use pairwise Mahalanobis distances or Euclidean distances?

I have the historic values of economic variables at quarter-ends for around 120 quarters. I wish to quantify how different the economy is at any two given point in time in my dataset (i.e. to produce ...
3
votes
0answers
111 views

Does maximal Bhattacharyya coefficient imply mimimal total variation distance?

I'm working on numerical optimization (linear programming), on probability distributions denoted P, Q. We want to find the minimal total variation distance and maximal Bhattacharyya coefficient ...
3
votes
0answers
444 views

Using similarity matrix to measure diversity of a group

I wish to measure the "diversity" of a group of objects. Right now I'm using Euclidean distance to compute the similarity matrix between all the objects in the group. I'm searching for a measure of ...
3
votes
0answers
323 views

Correct variance for minimum detectable difference

I have a question regarding variance, paired testing and minimum detectable difference (MDD). Paired samples: $$ MDD (δ) = \sqrt{ \frac{σ^2}{n} (t_{(α/2,n-1)} + t_{(1-β, n-1)})} $$ I have a set of ...
3
votes
0answers
280 views

Comparing model fit with heteroscedastic data

I am developing a physiological test using R that requires some parameters optimised. In comparing the new method against the existing method, the values of individual readings correlate in a linear ...
3
votes
0answers
405 views

Similarity measure for weighted sets (multisets)

I have to compute a similarity measure between different sets (Actually they are more like maps than sets). A weight is associated to each element of the set. The sets I want to compare represent ...
2
votes
1answer
23 views

Dice Distance returning nan. Workaround?

Starting Point: I want to calculate the distances between nominal data. Furthermore, I want to see what difference it makes to only use important features (given some feature selection method). So I ...
2
votes
1answer
31 views

Probability of points less than a fixed distance apart in a vector space

I have a distribution $D$ of points in a normed vector space (it's $\mathbb{R}^n$ using the $L_\infty$ norm, but I don't think that matters). In this particular space, points that are less than a ...
2
votes
0answers
47 views

Best approximation of the Mahalanobis distance by standardized Euclidean distance

I am looking for the best way to approximate the Mahalanobis distance by the standardized Euclidean distance, which would reduce the number of the required multiplications. The easiest way is the ...
2
votes
0answers
41 views

Detection Function in Distance Sampling, R

I'm fitting detection functions on whale data using the ds function in Distance package in R. I am having difficulty choosing the best detection function to then use in a density surface model. I ...
2
votes
0answers
77 views

Distance metric between structured 2D scenes

I am working on a problems that consist of simple 2D scenes, a typical example being a top-down traffic situation encountered by an autonomous vehicle. These structured scenes have multiple actors ...
2
votes
0answers
31 views

Spatio-temporal cluster analysis : take accessibility into consideration

I'm new in clustering topic. I have a data set with column A : Date, B: Event (let say a number of cases of any disease) and col C and D : Lat and Long respectively. And column E, distance-time ...
2
votes
0answers
32 views

Determine outliers for robust Mahalanobis distance

I want to apply a robust mahal distance and found an implementation in scikit: https://scikit-learn.org/stable/auto_examples/covariance/plot_mahalanobis_distances.html but there is the number of ...
2
votes
0answers
1k views

Hierarchical clustering dendrogram on a distance matrix

When computing hierarchical clustering over a data matrix, a dissimilarity matrix is first computed in order to build the tree (dendrogram). For example: ...
2
votes
0answers
579 views

Obtain within-group Gram matrix out of distance matrix

Gram matrix Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix ...
2
votes
0answers
535 views

Distance vs Dissimilarity measure

I am reading up on distance and dissimilarity measures for my class on natural language processing and could not understand this slide. Why does the dissimilarity measure not satisfy item 3 ? What ...
2
votes
0answers
348 views

Solving a system of equation by moment condition reports error but minimum distance works

I do not understand the following error message I get using the gmm function in R. The code below creates two moment conditions (...
2
votes
2answers
124 views

Inferring locations from groupings of items

Let's say I have a large series of three-item bundles, like so: ...
2
votes
0answers
263 views

“PCA” based on distance metric other than $L_2$

PCA is based on $L_2$ distance and is maximizing variance along the PC axes. What if we try a different distance measure (something else than $L_2$)? Do any methods corresponding to PCA but with ...
2
votes
0answers
792 views

How to measure distance between time series?

Given a quite noisy reference time-series $A$ (about 10k observations), and a bunch of equally sampled time-series $K^i$, is it possible to classify the $K^i$ according to their proximity to $A$? In ...
2
votes
0answers
136 views

Association between two asymmetric matrices

I have two matrices which elements are distances, created from an anisotropic cost analysis algorithm, between sites/populations. ...
2
votes
0answers
68 views

Practical application for theoretical sampling process

Does the following sampling process model any particular real life scenario ? If so, could you point me to some relevant scientific papers. Consider an arbitrary probability distribution $\mathcal{D}$...
2
votes
0answers
1k views

Cross-validated Manhattan/L1 distance

Consider the task that we want to compute the Euclidean distance between two vectors $\mathbf{a}$ and $\mathbf{b}$, where the vectors are noisy sample from some measurement. Our goal is to get an ...
2
votes
0answers
625 views

Looking for measures of similarity for two matrices of pairwise similarities / dissimilarities from different data

Hello CV World, my first time as OP, a little nervous here :P I'm looking for some statistical advice on the following problem. I speak python and R. I have two sets of data. These are matrices of ...

1
2 3 4 5 6