# Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

671 questions
Filter by
Sorted by
Tagged with
105k views

### Bottom to top explanation of the Mahalanobis distance?

I'm studying pattern recognition and statistics and almost every book I open on the subject I bump into the concept of Mahalanobis distance. The books give sort of intuitive explanations, but still ...
24k views

### Intuition on the Kullback–Leibler (KL) Divergence

I have learned about the intuition behind the KL Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The source I am reading goes on to say ...
88k views

### Choosing the right linkage method for hierarchical clustering

I am performing hierarchical clustering on data I've gathered and processed from the reddit data dump on Google BigQuery. My process is the following: Get the latest 1000 posts in /r/politics ...
52k views

### Maximum Mean Discrepancy (distance distribution)

I have two data sets (source and target data) which follow different distributions. I am using MMD - that is a non-parametric distribution distance - to compute marginal distribution between the ...
8k views

### Earth Mover's Distance (EMD) between two Gaussians

Is there a closed-form formula for (or some kind of bound on) the EMD between $x_1\sim N(\mu_1, \Sigma_1)$ and $x_2 \sim N(\mu_2, \Sigma_2)$?
50k views

12k views

### What is the purpose of row normalization

I understand the reasoning behind column normalization, as it causes features to be weighted equally, even if they are not measured on the same scale - however, often in the nearest neighbour ...
34k views

### Cosine Distance as Similarity Measure in KMeans [duplicate]

I am currently solving a problem where I have to use Cosine distance as the similarity measure for k-means clustering. However, the standard k-means clustering package (from Sklearn package) uses ...
2k views

### Is there an intuitive characterization of distance correlation?

I've been staring at the wikipedia page for distance correlation where it seems to be characterized by how it can be calculated. While I could do the calculations I struggle to get what distance ...
37k views

### Which distance to use? e.g., manhattan, euclidean, Bray-Curtis, etc

I am not a community ecologist, but these days I am working on community ecology data. What I couldn't understand, apart from the mathematics of these distances, is the criteria for each distance to ...
4k views

### Is there any probability distance that preserves all properties of a metric?

In studying Kullback–Leibler distance, there are two things we learn very quickly is that it does not respect neither the triangle inequality nor the symmetry, required properties of a metric. My ...
4k views

### Why is Kullback-Leilbler divergence a better metric for measuring distance between two probability distributions than squared error? [duplicate]

I know that KL-divergence is a metric that is more suitable when we want to measure the distance between numbers which a probability form. However, I am still confused what is the benefit of using KL-...
7k views

### Is triangle inequality fulfilled for these correlation-based distances?

For hierarchical clustering I often see the following two "metrics" (they aren't exactly speaking) for measuring the distance between two random variables $X$ and $Y$: $\newcommand{\Cor}{\mathrm{Cor}}$...
11k views

### Intuition of the Bhattacharya Coefficient and the Bhattacharya distance?

The Bhattacharyya distance is defined as $D_B(p,q) = -\ln \left( BC(p,q) \right)$, where $BC(p,q) = \sum_{x\in X} \sqrt{p(x) q(x)}$ for discrete variables and similarly for continuous random variables....
24k views

### How does the Gower distance calculate the difference between binary variables'?

I have 17 numeric and 5 binary (0-1) variables, with 73 samples in my dataset. I need to run a cluster analysis. I know that the Gower distance is a good metric for datasets with mixed variables. ...
19k views

### Distance Metrics For Binary Vectors

I have vectors of same length consisting of 1 and 0. I am trying to find out how similar they are. So far I am using hamming distance that I calculate sum of one vector then sum of second vector and ...
6k views

### Dynamic Time Warping for irregular time series

I have been reading a lot about Dynamic Time Warping (DTW) lately. I am very surprised that there is no literature at all on the application of DTW to irregular time series, or at least I could not ...
2k views

### Efficient/fast Mahalanobis distance computation

Suppose I have $n$ data points $x_1,\dots,x_n$, each of which is $p$-dimensional. Let $\Sigma$ be the (non-singular) population covariance of these samples. With respect to $\Sigma$, what is the most ...
3k views

### Statistical significance of difference between distances

I have over 3000 vectors on a two-dimensional grid, with an approximately uniform discrete distribution. Some pairs of vectors fulfil a certain condition. Note: the condition is only applicable to ...
9k views

### Intuitively, why is cross entropy a measure of distance of two probability distributions?

For two discrete distributions $p$ and $q$, cross entropy is defined as $$H(p,q)=-\sum_x p(x)\log q(x).$$ I wonder why this would be an intuitive measure of distance between two probability ...
4k views

### What is the distance between a finite Gaussian mixture and a Gaussian?

Suppose I have a mixture of finitely many Gaussians with known weights, means, and standard deviations. The means are not equal. The mean and standard deviation of the mixture can be calculated, of ...
559 views

### Does Mercer's theorem work in reverse?

A colleague has a function $s$ and for our purposes it is a black-box. The function measures the similarity $s(a,b)$ of two objects. We know for sure that $s$ has these properties: The similarity ...
1k views

### Finding a known number of circle centers that maximize the number of points within a fixed distance

I have a set of 2-D data where I want to find the centers of a specified number of centers of circles ($N$) that maximize the total number of points within a specified distance ($R$). e.g. I have 10,...
4k views

### How to compare two distance matrices?

Suppose that I have two distance matrices for the same set of items. By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
10k views

### What are distances between variables making a covariance matrix?

I have a $n \times n$ covariance matrix and want to partition variables into $k$ clusters using hierarchical clustering (for example, to sort a covariance matrix). Is there a typical distance ...
18k views

### Similarity measure between multiple distributions

To compare distributions, it is common to use box blots. I'm looking for a similarity measure that calculates whether distributions are the similar or not. Ideally, given that e.g. four ...
10k views

### Is Mahalanobis distance equivalent to the Euclidean one on the PCA-rotated data?

I've been led to believe (see here and here) that Mahalanobis distance is the same as the Euclidean distance on the PCA-rotated data. In other words, taking multivariate normal data $X$, the ...
9k views

### How to find the expected distance between two uniformly distributed points?

If I were to define the coordinates $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ where $$X_{1},X_{2} \sim \text{Unif}(0,30)\text{ and }Y_{1},Y_{2} \sim \text{Unif}(0,40).$$ How would I find the expected ...
3k views

### Clustering with asymmetrical distance measures

How do you cluster a feature with an asymmetrical distance measure? For example, let's say you are clustering a dataset with days of the week as a feature - the distance from Monday to Friday is not ...
2k views

### Distance metric and curse of dimensions

Some where I read a note that if you have many parameters $(x_1, x_2, \ldots, x_n)$ and you try to find a "similarity metric" between these vectors, you may have a "curse of dimensioality". I believe ...
4k views

### Probability that uniformly random points in a rectangle have Euclidean distance less than a given threshold

Assume we have $n$ points in a rectangular with bound $[0,a] \times [0,b]$, and these points are uniformly distributed in this plane. (I am not quite familiar with statistics, so I don't know the ...
3k views

### Expected magnitude of a vector from a multivariate normal

What is the expected magnitude, i.e. euclidean distance from the origin, of a vector drawn from a p-dimensional spherical normal $\mathcal{N}_p(\mu,\Sigma)$ with $\mu=\vec{0}$ and $\Sigma=\sigma^2 I$, ...
22k views

### K-means: Why minimizing WCSS is maximizing Distance between clusters?

From a conceptual and algorithmic standpoint, I understand how K-means works. However, from a mathematical standpoint, I don't understand why minimizing the WCSS (within-cluster sums of squares) will ...
7k views

### How is the distance formula related to the formula for standard deviation

The formula for the standard deviation of n numbers is the same as the formula for the distance between two points in n dimensions. Could someone explain why this is and how these are related?