Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

671 questions
Filter by
Sorted by
Tagged with
105k views

Bottom to top explanation of the Mahalanobis distance?

I'm studying pattern recognition and statistics and almost every book I open on the subject I bump into the concept of Mahalanobis distance. The books give sort of intuitive explanations, but still ...
• 5,257
24k views

Intuition on the Kullback–Leibler (KL) Divergence

I have learned about the intuition behind the KL Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The source I am reading goes on to say ...
• 7,815
88k views

Choosing the right linkage method for hierarchical clustering

I am performing hierarchical clustering on data I've gathered and processed from the reddit data dump on Google BigQuery. My process is the following: Get the latest 1000 posts in /r/politics ...
• 611
52k views

Maximum Mean Discrepancy (distance distribution)

I have two data sets (source and target data) which follow different distributions. I am using MMD - that is a non-parametric distribution distance - to compute marginal distribution between the ...
• 431
8k views

Earth Mover's Distance (EMD) between two Gaussians

Is there a closed-form formula for (or some kind of bound on) the EMD between $x_1\sim N(\mu_1, \Sigma_1)$ and $x_2 \sim N(\mu_2, \Sigma_2)$?
• 391
50k views

• 821
12k views

What is the purpose of row normalization

I understand the reasoning behind column normalization, as it causes features to be weighted equally, even if they are not measured on the same scale - however, often in the nearest neighbour ...
34k views

Cosine Distance as Similarity Measure in KMeans [duplicate]

I am currently solving a problem where I have to use Cosine distance as the similarity measure for k-means clustering. However, the standard k-means clustering package (from Sklearn package) uses ...
• 255
2k views

Is there an intuitive characterization of distance correlation?

I've been staring at the wikipedia page for distance correlation where it seems to be characterized by how it can be calculated. While I could do the calculations I struggle to get what distance ...
• 6,482
37k views

Which distance to use? e.g., manhattan, euclidean, Bray-Curtis, etc

I am not a community ecologist, but these days I am working on community ecology data. What I couldn't understand, apart from the mathematics of these distances, is the criteria for each distance to ...
• 355
4k views

Is there any probability distance that preserves all properties of a metric?

In studying Kullback–Leibler distance, there are two things we learn very quickly is that it does not respect neither the triangle inequality nor the symmetry, required properties of a metric. My ...
• 1,249
4k views

Why is Kullback-Leilbler divergence a better metric for measuring distance between two probability distributions than squared error? [duplicate]

I know that KL-divergence is a metric that is more suitable when we want to measure the distance between numbers which a probability form. However, I am still confused what is the benefit of using KL-...
• 385
7k views

Is triangle inequality fulfilled for these correlation-based distances?

For hierarchical clustering I often see the following two "metrics" (they aren't exactly speaking) for measuring the distance between two random variables $X$ and $Y$: $\newcommand{\Cor}{\mathrm{Cor}}$...
• 133
11k views

Intuition of the Bhattacharya Coefficient and the Bhattacharya distance?

The Bhattacharyya distance is defined as $D_B(p,q) = -\ln \left( BC(p,q) \right)$, where $BC(p,q) = \sum_{x\in X} \sqrt{p(x) q(x)}$ for discrete variables and similarly for continuous random variables....
• 3,949
24k views

How does the Gower distance calculate the difference between binary variables'?

I have 17 numeric and 5 binary (0-1) variables, with 73 samples in my dataset. I need to run a cluster analysis. I know that the Gower distance is a good metric for datasets with mixed variables. ...
• 289
19k views

Distance Metrics For Binary Vectors

I have vectors of same length consisting of 1 and 0. I am trying to find out how similar they are. So far I am using hamming distance that I calculate sum of one vector then sum of second vector and ...
• 241
6k views

Dynamic Time Warping for irregular time series

I have been reading a lot about Dynamic Time Warping (DTW) lately. I am very surprised that there is no literature at all on the application of DTW to irregular time series, or at least I could not ...
• 275
2k views

Efficient/fast Mahalanobis distance computation

Suppose I have $n$ data points $x_1,\dots,x_n$, each of which is $p$-dimensional. Let $\Sigma$ be the (non-singular) population covariance of these samples. With respect to $\Sigma$, what is the most ...
3k views

Statistical significance of difference between distances

I have over 3000 vectors on a two-dimensional grid, with an approximately uniform discrete distribution. Some pairs of vectors fulfil a certain condition. Note: the condition is only applicable to ...
• 123
9k views

Intuitively, why is cross entropy a measure of distance of two probability distributions?

For two discrete distributions $p$ and $q$, cross entropy is defined as $$H(p,q)=-\sum_x p(x)\log q(x).$$ I wonder why this would be an intuitive measure of distance between two probability ...
• 121
4k views

What is the distance between a finite Gaussian mixture and a Gaussian?

Suppose I have a mixture of finitely many Gaussians with known weights, means, and standard deviations. The means are not equal. The mean and standard deviation of the mixture can be calculated, of ...
• 10.4k
559 views

Does Mercer's theorem work in reverse?

A colleague has a function $s$ and for our purposes it is a black-box. The function measures the similarity $s(a,b)$ of two objects. We know for sure that $s$ has these properties: The similarity ...
• 79.4k
1k views

Finding a known number of circle centers that maximize the number of points within a fixed distance

I have a set of 2-D data where I want to find the centers of a specified number of centers of circles ($N$) that maximize the total number of points within a specified distance ($R$). e.g. I have 10,...
• 659
4k views

How to compare two distance matrices?

Suppose that I have two distance matrices for the same set of items. By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
• 343
10k views

What are distances between variables making a covariance matrix?

I have a $n \times n$ covariance matrix and want to partition variables into $k$ clusters using hierarchical clustering (for example, to sort a covariance matrix). Is there a typical distance ...
• 5,616
18k views

Similarity measure between multiple distributions

To compare distributions, it is common to use box blots. I'm looking for a similarity measure that calculates whether distributions are the similar or not. Ideally, given that e.g. four ...
• 131
10k views

Is Mahalanobis distance equivalent to the Euclidean one on the PCA-rotated data?

I've been led to believe (see here and here) that Mahalanobis distance is the same as the Euclidean distance on the PCA-rotated data. In other words, taking multivariate normal data $X$, the ...
• 12.2k
9k views

How to find the expected distance between two uniformly distributed points?

If I were to define the coordinates $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ where $$X_{1},X_{2} \sim \text{Unif}(0,30)\text{ and }Y_{1},Y_{2} \sim \text{Unif}(0,40).$$ How would I find the expected ...
• 403
3k views

Clustering with asymmetrical distance measures

How do you cluster a feature with an asymmetrical distance measure? For example, let's say you are clustering a dataset with days of the week as a feature - the distance from Monday to Friday is not ...
• 2,220
2k views

Distance metric and curse of dimensions

Some where I read a note that if you have many parameters $(x_1, x_2, \ldots, x_n)$ and you try to find a "similarity metric" between these vectors, you may have a "curse of dimensioality". I believe ...
• 1,863
4k views

Probability that uniformly random points in a rectangle have Euclidean distance less than a given threshold

Assume we have $n$ points in a rectangular with bound $[0,a] \times [0,b]$, and these points are uniformly distributed in this plane. (I am not quite familiar with statistics, so I don't know the ...
3k views

Expected magnitude of a vector from a multivariate normal

What is the expected magnitude, i.e. euclidean distance from the origin, of a vector drawn from a p-dimensional spherical normal $\mathcal{N}_p(\mu,\Sigma)$ with $\mu=\vec{0}$ and $\Sigma=\sigma^2 I$, ...
• 302
22k views

K-means: Why minimizing WCSS is maximizing Distance between clusters?

From a conceptual and algorithmic standpoint, I understand how K-means works. However, from a mathematical standpoint, I don't understand why minimizing the WCSS (within-cluster sums of squares) will ...
• 484
7k views

How is the distance formula related to the formula for standard deviation

The formula for the standard deviation of n numbers is the same as the formula for the distance between two points in n dimensions. Could someone explain why this is and how these are related?
8k views

How to measure distance for features with different scales?

I'm reading the book "Collective Intelligence" and in one chapter they introduce how to measure similarity between users on a movie review website with euclidean distance. Now are the movies rated ...
• 263
7k views

Measure for Separability

I have a (binary) classification problem where after merging single training data points (that can be tracked back to the same source) into aggregates, test accuracy (on single data points again) ...
• 295