Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

0
votes
1answer
30 views

R: What is the distance function equivalent for this formula?

Hi I'm using an R package that calculates distance with this formula here, as.dist(1 - cor(df, use = "pa")) However I cannot seem to find an equivalent dist ...
0
votes
0answers
18 views

How to localize points from an incomplete distance matrix in R?

Suppose you have 3 shops and 2 supply units, and you only know the 6 pairwise (Euclidean, assuming 2D) distances between each shop and each supply unit, but not the pairwise distances between the ...
0
votes
0answers
3 views

Quick search based on similarity: logarithmic time

I have some objects $x\in X$ and a metric $s:X\times X\to\mathbb{R_{+}}$. For each $x$, there is a $y\in Y$. Note that $x$ and $y$ are highly structured and we cannot consider neural networks for ...
0
votes
0answers
7 views

Calculate association trends

I have time series dataset for 10 consecutive periods (i.e. T, T+1, T+2, ..., T+9). Moreover, I also have 100 term triplets in each time period. Each triplet contains 3 objects namely x, y and z. I ...
3
votes
1answer
40 views

Calculation of relative distance

I have 100 term triplets as shown in the below mentioned figure. Each triplet contains 3 objects namely x, y and z. I want to rank the triplets according to the following two properties. y should be ...
1
vote
1answer
52 views

Mahalanobis distance gives counterintuitive results [closed]

I have generated 100 sample time series, each 24 items long, and each with an exponential distribution with a different scale for each of the 24 time points. This is the scale parameter per time point:...
0
votes
0answers
33 views

Can Mahalanobis distance not be applied reliably after performing one-hot encoding with certain data-sets?

I am working with a data-set of patient performance-data and patient demographics for people with a medical condition. I am trying to assess the effect of a treatment on the patients, and as I am ...
0
votes
0answers
11 views

Variable transformation for converting Euclidean distance to Mahalanobis distance

I'm working on a project that involves computing distances for correlated variables with dissimilar variances for approximate Bayesian computation. Sounds like a job for Mahalanobis distances. ...
2
votes
1answer
45 views

Basic Hellinger Distance

One definition of Hellinger Distance is $$L_{H}: (1/2) E_{\theta}([\sqrt{\frac{f(x|\delta)}{f(x|\theta)}}-1])^{2}$$ My book has that for $x \sim N(\theta,1)$ $$L_{H}(\theta,\delta)=1-\exp(-(1/8)(\...
1
vote
0answers
30 views

Mahalanobis distance between 2 points doesn't work when covariance matrix has values close to 0

I am working on a project where I am trying to replicate a randomized experiment from an observational study data, using Mahalanobis distance matching to ensure that the control and treated groups are ...
2
votes
1answer
40 views

How can I order kmeans clusters?

I have a kmeans cluster object and I would like to order the clusters. Not the observations within the clusters, rather the clusters in order of each other. Is there a way of doing this? I found ...
0
votes
1answer
20 views

Kmeans results, is the cluster vector ordered by 'closeness"?

I ran kmeans in r with k = 20 centers and 7 scaled variables to cluster with on a data frame with n = 100K. Using dplyr group_by I was able to view summary data for each of the 20 clusters: the mean ...
-1
votes
1answer
27 views

How k-means computes cluster centroids differently for each distance metric?

K-means computes cluster centroids differently for each distance metric. I don't know why the way of computing the centroid is dependent of the distance measure. I don't know how we compute the ...
0
votes
0answers
12 views

Robust distance measure for correlated data

I read a paper in which the authors want to compare the overall predictive accuracy of various predictors on a set of variables by using the Mahalanobis-Distance. However the data is not even ...
0
votes
0answers
18 views

what is the mathematical difference between the distances when clustering text

Suppose i have data for text clustering, there 300 000 rows text$GOODS_NAME let's do text clustering. ...
0
votes
0answers
12 views

Methods to minimize distance between features

What are some methods to minimize distances between features belonging to a cluster? I want to do this regardless of their class. Say I have a $m\times d$ feature matrix of $m$ samples, each with $d$ ...
3
votes
2answers
131 views
0
votes
1answer
20 views

Clustering by same random projection

I have $N$, $1024$-dimensional vectors. I want to cluster them by some similarity. Given the high dimensionality, standard metrics won't work. I tried a few Approximate Nearest Neighbor ...
1
vote
0answers
36 views

Compare KS test and Wasserstein distance or Earth mover's distance

Consider two sets of data points A and B. Both these data points are from mixture of unknown number of Gaussians. The mean of the Gaussians are little different for each set (there may have few ...
0
votes
0answers
14 views

Comparison of empirical discrete distributions. Pros and cons of different metrics?

I am trying to measure the dissimilarity between two empirical discrete distributions. I am aware of various distance metrics that could be used for this purpose such as Wasserstein, Bhattacharyya etc....
1
vote
0answers
15 views

Mean distance from the centre to any point in a sphere and a cylinder [closed]

What is the mean distance from the centre to any point within a sphere of radius r? What is the mean distance from the centre to any point within a cylinder of radius r and length l?
0
votes
0answers
37 views

How to quantify distance between 2 datasets?

I have a distribution $A$ (intent-to-treat population) and its subset $B \subset A$ (treated population). I learn a propensity model $P(x \in B)$ to predict treatment. Then I sample the intent-to-...
1
vote
0answers
48 views

Distance between angle distributions

I want to quantify the complexity of the street network of different cities. For each city I have the angle distribution of its streets. My hypothesis that the more complex the street network, the ...
0
votes
0answers
38 views

Distance between time series and representation (Piecewise Aggregate Approximation - PAA) (R)

I am trying to come up with a way of measuring the distance between a time series and its representation in order to see how closely the representation describes the original series. I feel like there ...
0
votes
0answers
22 views

How to find Mahalanobis distance between an observation and a population with mixed data?

I have a dataset(let's call it as 'D') with multiple continuous, nominal and ordinal variables, as follows: continuous: Total sales as well as sales figures of some products by customers nominal: ...
0
votes
0answers
26 views

Comparing distances evaluated on different vector spaces

We have a dataset of I items who have been measured over two different sets of features A, with cardinality N, and B with cardinality M, and N > M. We would like to know in which feature space the ...
3
votes
3answers
186 views

Probability that an observation comes from population A or B?

I'm a web developer looking into some basic statistics -- pardon me if I am using the wrong jargon. :) Considering that: I have 2 populations (A and B; each have about 10,000 observations) For each ...
0
votes
1answer
40 views

Hellinger distance for two shifted log-normal distributions

If I am not mistaken, Hellinger distance between P and Q is generally given by: $$ H^2(P, Q) = \frac12 \int \left( \sqrt{dP} - \sqrt{dQ} \right)^2 .$$ If P and Q, however, are two differently ...
0
votes
0answers
12 views

How to Measure Similarity to Ground Truth? (Measuring Similarity with Features of Different Scales/Units)

I am trying to reproduce a set of ground truth data [t_start | t_end | theta_start | theta_end] (blue) A plot of my ground truth data would show a scatter of lines along the time and angular axes....
2
votes
1answer
60 views

Choosing appropriate distance metric and algorithm for clustering for any given dataset

I have been looking for an answer/guidance/pointer to this question of mine for a while. After going through many (100s actually) posts and articles, I finally found this question, where this response ...
2
votes
1answer
41 views

Distribution of distance of N-1 gamma distributed iid random variables from minimum

I have the minimum value of N iid random variables that are gamma-distributed. The parameters of the gamma distribution are known. What would be the distribution of the distance of the remaining N - 1 ...
0
votes
1answer
26 views

detect an outlier in multi-dimensions - where the number of rows is not >> number of columns

I have the following situation: a data frame with two dimensions x and y, with three "areas": an a-parametric distribution ...
1
vote
0answers
63 views

Is pairwise distance matrix useful to k-means?

The k-means implemented in scikit-learn precomputes distances but I don't how these distances are used. In its standard version, k-means is known to compute only the distances between the points and ...
5
votes
1answer
270 views

Suitable distance metric for time-series clustering with respect to location of shapes

I'm doing clustering on time-series (each time-series has the information for one day = 24 hours). For the clustering purpose, it's important for me to consider the time period in which the shape of ...
2
votes
0answers
127 views

Obtain within-group Gram matrix out of distance matrix

Gram matrix Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix ...
0
votes
0answers
31 views

How to compute distances with both categorical and continuous attributes?

I have to handle with a datast containing both categorical attributes (around 25) and continuous attributes (around 25). I would like to do outliers detection. I think that it would be a good idea to ...
0
votes
1answer
16 views

Single Linkage Clustering with Manhattan metric

Say suppose we are having 5 data points with 3 attributes each ... (4,3,1) (2,1,5) (1,2,3) (2,3,1) .... Now let us build the distance matrix. If we do Manhattan metric then the cell corresponding to ...
1
vote
0answers
61 views

What happens with Mahalanobis-Distance, when the assumption of equal Covariance-Matrices breaks down

Assume that we want to compare the forecast quality of various forecasters $f$ on $n$ values such as stock-market prices or whatever. We could then define a "Mahalanobis-Distance" (MD) (or rather ...
0
votes
0answers
30 views

Common methods to calculate total distance from data with categorical, continuous and counting variables

I have a data set with categorical, continuous and counting variables. I want to be able to use a method that will give me a distance for each pairwise data point. From my understanding, each type ...
0
votes
0answers
40 views

Similarity of two sets of points

PROBLEM I have one set of 10 points, $X = \{(x_1,y_1),\,\dots,\,(x_{10},y_{10})\}$ and two sets of 3 points each, $A = \{ (a_1,b_1),\, (a_2,b_2),\, (a_3,b_3) \}$ and $C = \{(c_1,d_1),\,(c_2,d_2),\,(...
0
votes
0answers
18 views

Which is the best distance metric in an Indicator matrix

Is it okay to use the $\chi^2-distance$ when we have a indicator matrix? With Indicator matrix I mean the complete dijuntive table that is used in the Multiple Correspondence analysis. I mea n, if we ...
0
votes
1answer
83 views

Is Earth Mover Distance has maximum bound?

I have two probability distributions which each distribution has sum up to 1. I want to compute the distance between those two probability distributions. I want to use Earth Mover Distance to ...
4
votes
2answers
110 views

The Curse of high Dimension And Distance

For extracting features from video frames (2 sample/sec) I use keras framework in python and load VGG16 that input size is (150,150,3) and output size is (4,4,512). After the feature extraction step I ...
4
votes
2answers
104 views

Is the maximum bound of Euclidean distance between two probability distributions equal to $\sqrt{2}$?

I used Euclidean distance to compute the distance between two probability distribution. The example of computation shown in the Figure below. As my understanding, the maximum distance occur while $...
11
votes
3answers
2k views

What's the maximum value of Kullback-Leibler (KL) divergence

I am going to use KL divergence in my python code and I got this tutorial. On that tutorial, to implement KL divergence is quite simple. ...
6
votes
2answers
297 views

Do Autoencoders preserve distances?

Based on my understanding, autoencoders are used to find a compact representation of input features that carries the essential underlying information. Is there any relationship between the L2 ...
1
vote
0answers
85 views

Gower's Distance + PCA : How to interpret in R? [closed]

I have a dataset (40 records and 10 variables that is a combination of continuous variables and categorical variables (age, work experience, current salary, educational qualification, house in rural/...
0
votes
0answers
56 views

On the right dissimilarity matrix for discrete cluster analysis

Currently I'm having some trouble on deciding the right dissimilarity matrix for my data. After reading some previously asked questions here, I'm even left more confused. So, here is the deal. My data ...
0
votes
0answers
17 views

How well does a subsample of a large dataset represents the full set in a statistical sense?

Suppose that a large sample $ \{X_k\}_{k=1}^{n} $ (from a multivariate distribution $ X $) is given. I would like find a subsample $ \{ Y_k \}_{k=1}^{m} \subset \{X_k\}_{k=1}^{n} $ of this dataset in ...
1
vote
0answers
20 views

Distance vs Dissimilarity measure

I am reading up on distance and dissimilarity measures for my class on natural language processing and could not understand this slide. Why does the dissimilarity measure not satisfy item 3 ? What ...