Questions tagged [distance]
Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.
537
questions
1
vote
0answers
15 views
How to use Gower's Distance with DBSCAN algorithm in Python
I have been researching about using DBSCAN with sklearn in python but it doesn't have Gower's distance metric built in. All the other implementations are in R in this community.
I'm using a dataset ...
0
votes
0answers
38 views
Manhattan vs Euclidian Distance Measure [duplicate]
In which case we should pickup Manhattan distance and when we should use euclidian distance measure.
To my understanding both are used for continues numeric data(not like cosine or others who works ...
0
votes
0answers
17 views
Can the ranks of summary statistics discriminate between (shapes of) multiple distributions? (intuition)
TL;DR
Can we say the following statement, especially the first part of it? It feels intuitively true:
(1) The ranks of summary statistics characterize the overall shape of distributions... (2) ...
1
vote
2answers
39 views
Understanding the R stats mahalanobis() function's Output
An acquaintance recommended I use the Mahalanobis distance on my data instead of Euclidean, Manhattan, etc.
I tried using the mahalanobis() function in the R stats package on a data matrix with N ...
2
votes
0answers
23 views
Spatio-temporal cluster analysis : take accessibility into consideration
I'm new in clustering topic. I have a data set with column A : Date, B: Event (let say a number of cases of any disease) and col C and D : Lat and Long respectively. And column E, distance-time ...
0
votes
1answer
55 views
In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbours. Why not manhattan distance?
In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbours. Why not manhattan distance ?
1
vote
0answers
14 views
Determine outliers for robust Mahalanobis distance
I want to apply a robust mahal distance and found an implementation in scikit: https://scikit-learn.org/stable/auto_examples/covariance/plot_mahalanobis_distances.html
but there is the number of ...
0
votes
1answer
25 views
How to find the list of nearest vectors if ony a vector is given?
I know there are many ways to compute similarity of two different non-zero vectors but is it possible to get a list of nearest vectors whose values are continous given a single continous vector.
Lets ...
0
votes
0answers
14 views
Clustering block matrices
The question is about clustering (or finding the distance of) the submatrices of a matrix in the presence of block missings. Starting by the example in R. Let ...
0
votes
0answers
27 views
Understanding the kdist graph used to select DBSCAN epsilon parameter
I need to use DBSCAN for my research and am having trouble understanding the kdist graph used to select the epsilon parameter - specifically, I do not understand what is happening behind the scenes ...
0
votes
0answers
25 views
Maximum likelihood as minimizing the dissimilarity between the empirical distriution and the model distribution
I am reading Ian Goodfellow "Deep Learning" book. At page 128 it says
One way to interpret maximum likelihood estimation is to view it as
minimizing the dissimilarity between the empirical ...
0
votes
0answers
7 views
Measuring the variability of a distance measure vector
I have calculated the distance of data points according to the Maalanobis distance. Now I have a vector of distances that I am trying to measure its variability to identify the residuals. I was ...
0
votes
1answer
15 views
Distance metrics with missing data where the missing data are informative
I am attempting to cluster subgroups of substance abuser based on diagnostic status (nominal), age of onset (ordinal since it is binned in our set), etc. My question regards how to treat missing data ...
0
votes
0answers
10 views
Multiple correspondence analysis, definition of distance between two categories of the same question
From the text : Multiple Correspondents Analysis by Brigette LeRoux
The data for this quesiton is:
The definition of $f_k$ is
$f_k = n_k/n$ where $n$ is the total number of individuals and $n_k$ ...
1
vote
1answer
28 views
Measuring variation explained in community matrix using both geographic distance and and environmental variables
So I have this dataset where I have species community data from a variety of sites. I’m trying to explain what are the factors that drive the variation in these data. For each site, I have a ...
0
votes
1answer
42 views
In VEGAN ADONIS, what does "method='bray' do when I pass in a distance matrix I already have (UNIFRAC)?
Suppose I have a distance matrix created by applying the weighted unifrac measure. For instance, using a phyloseq object:
...
0
votes
1answer
25 views
Correlation between two distances (distance matrixes)?
Premise: I have a dataset of elements for which I have a representation in 2 different spaces, a "latent" space and the original space (I can move between those with an Autoencoder neural network).
I ...
0
votes
1answer
24 views
Mahalanobis distance - understanding the formula [duplicate]
I've read quite a few explanations on this topic, liking this one the most:
https://mccormickml.com/2014/07/22/mahalanobis-distance/
But there is still one thing I don't understand.
I understand ...
0
votes
0answers
18 views
Type of logarithm in Jenson-Shannon and Bhattacharyya distance
Both Jenson-Shannon and Bhattacharyya distance can be used to measure the similarity of two probability distributions.
Bhattacharyya distance between two distributions $p$ and $q$ is defined as $D_B(...
0
votes
0answers
24 views
Truncation of distance matrix
Given a sample $\textbf{X}_1$,...,$\textbf{X}_n\in\mathbb{R}^p$ from an arbitrary distribution with distribution function $F$ we can calculate the pairwise Mahalanobis distances between the sample ...
0
votes
0answers
12 views
Question About Coming Up With Own Function for Distance Matrix (For Clustering)
Right now, I am currently working on implementing a clustering algorithm with millions data entries with regards to game users for a mobile game.
A lot of the features I plan on using are unique to ...
0
votes
1answer
39 views
When a distance in Dynamic Time Warping is said to be 'short' or 'optimal'?
I'm doing a research on Dynamic Time Warping and I wasn't able to find a certain number to be considered an optimal distance.
I wonder if there exists some value or it depends on the datasets ...
0
votes
0answers
35 views
Weigh Kullback Leibler Divergence with P entropy?
I'm wondering if it makes sense to weight Kullback-Leibler Divergences to highlight divergences on highly distinguishing features. I am however not very well versed in Information Theory, and would ...
14
votes
3answers
2k views
How to measure the statistical “distance” between two frequency distributions?
I am undertaking a data analysis project which involves investigating website usage times over the course of the year. What I would like to do is compare how "consistent" the usage patterns are, say, ...
0
votes
0answers
16 views
Calculating UniFrac distance
So I am trying to calculate unifrac distance for my OTU table, which has about 3000 taxa, 300 samples. Now my phylogenetic tree only consider the most important 500 taxa. My questions are:
Do I ...
0
votes
0answers
41 views
How to tell if a set of MCMC chains mixed?
Let's say I have a Bayesian network with both numeric and categorical variables. I run several MCMC chains to collect samples from the distribution. Now, if the chains are "similar enough" after some ...
0
votes
0answers
30 views
How is the mahalanobis distance like the euclidean distance? [duplicate]
Let's say $\vec{x}$ is an $n$ dimensional observation, $\vec{\mu}$ the $n$ dimensional mean of the sample that $\vec{x}$ is from and $\Sigma$ the $n \times n$ covariance matrix of that sample.
Then ...
0
votes
1answer
16 views
Distance metric for sequential spatial data (routes navigated in 2d space)
I'm looking for a distance metric to compute how close certain paths taken by people navigating throughout a city are to a set of 'correct' routes.
I have path recordings for some 'correct' routes ...
1
vote
1answer
31 views
L1 distance between categorical distribution and any arbitrary estimator?
Given an unknown categorical distribution $p$ over $k$ categories, and any arbitrary estimator of this distribution vector $q$ constructed from $n$ i.i.d samples, can anyone point me to some results ...
0
votes
1answer
45 views
distance metric for student course schedules
I'm doing an exploratory clustering analysis of student course schedules at a college. Interpretability by humans is paramount: we're trying to inform future research questions and possibly ...
3
votes
1answer
48 views
What's the distribution of the closest point from uniform samples?
Suppose you have $N$ values $x_1, \ldots, x_N$ that are uniformly sampled in $[0; 1]$. For a random $x_k$ amongst the $(x_i)_i$ (with equiprobability), what is the expected value of the distance ...
1
vote
0answers
32 views
Wasserstein distance / EMD of two sets of 2D weighted points?
I am trying to implement a 2D version of the EMD/Wasserstein Distance to measure the distance of sets of 2D weighted points.
Let $A = \{a_{1}, a_{2}, ..., a_{m}\}$, be a weighted point set such that $...
0
votes
0answers
28 views
Calculate Distance between two vectors and estimate goodness of fit to preestablished histogram shapes
I have a squared gene co-expression correlation matrix of many thousands of pairwise correlations among variables (10290^2 aprox). Each row/column represents a different gene and its pairwise ...
0
votes
0answers
38 views
Compare statistical distances of multiple distributions
I need a metric that not only gives me the statistical distance between two distributions, but that also is comparable to another distance between two completely different distributions, calculated ...
0
votes
0answers
56 views
Can Bhattacharyya coefficient (or distance) be used as an additive measure to compute a metric for performance?
As far as I understand, Bhattacharyya's measure(s) can be used to see similarity between two empirical distributions. Other ways to do so are nicely explained here: Similarity measure between multiple ...
0
votes
0answers
21 views
Learning distance metric from output of knn
Consider a set $\mathcal X$ of points $\{x_1,\dots,x_n,x_{n+1},\dots,x_{n+m} \}\subset \mathbb R^p$. Let $A$ be some $p\times p$ matrix, unknown to you. Consider the set $$\mathcal X_A:=\{y_1,\dots,y_{...
0
votes
0answers
27 views
Distance between objects with different attributes meaningful?
Let's say you have two sets of apples and you assign some attributes to them like color, size...
0
votes
0answers
61 views
Difference between standardizing variables and using Mahalanobis distance
I am wondering how and/or why the Mahalanobis distance is different from using the Euclidean distance on standardized variables?
1
vote
0answers
42 views
How does one apply hierarchical agglomerative clustering when multiple positions are equidistant (non-unique distance matrix)?
I have been following this single/minimum -linkage example to better understand hierarchical agglomerative clustering. I noticed that the entries of d_ij are unique ...
0
votes
0answers
14 views
Is it correct to use “Ward.D2” 's method of R's hclust function with a Gower distance matrix? [duplicate]
I have mixed type variables (3 quantitative and 3 qualitative) and I calculated Gower's dissimilarity distance between my objects. I wanted to do a hierarchical clustering with hclust, but I am not ...
1
vote
1answer
44 views
Something like Mahalanobis distance when the copula is not Gaussian
Mahalanobis distance accounts for different variances of the marginal variables and correlations between the marginal variables. However, there is an implicit (maybe explicit) assumption that ...
3
votes
0answers
28 views
Distribution $f$ that minimizes $JSD(f||q) + JSD(f||p)$
What can we say about the distribution $f^*$ that is the solution to the following optimization problem:
$$\min_f JSD(f||p)+JSD(f||q) ,$$
where $p,q$ are given distributions over some set, and $JSD$ ...
0
votes
0answers
3 views
Comparing which histogram has overall low cost
Let's say there are two histograms which basically is constructed from array of numbers which is measured by, repeatedly performing a task by two different methods and individual numbers are time ...
9
votes
1answer
912 views
Why is Kullback-Leilbler divergence a better metric for measuring distance between two probability distributions than squared error? [duplicate]
I know that KL-divergence is a metric that is more suitable when we want to measure the distance between numbers which a probability form. However, I am still confused what is the benefit of using KL-...
2
votes
0answers
116 views
Relation Between Wasserstein Distance and Relative Entropy
Consider the Wasserstein metric of order one $W_1$ (aka the Earth Movers Distance). I would like to know whether it is possible to link $W_1$ and relative entropy and what this would mean intuitively. ...
0
votes
0answers
10 views
Distance correlation and a corresponding mapping
I have two long vectors, say X and Y (of equal length). I computed the Distance Correlation as implemented in Scipy and I got a ...
0
votes
0answers
45 views
Signature of a distribution - Earth Mover distance
I am studying the Earth Mover Distance from here, but I have some difficulty in fully understanding what is the signature of a distribution and how it matches with the last constraint of the Earth ...
0
votes
0answers
71 views
Total variation norm
I am reading
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012, January). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science ...
0
votes
1answer
30 views
estimate equal distribution of few points on a line
I am trying to find the best solution to estimate equal distribution of points over a line. I know I can use relative SD or similar, but I was wondering if there are more "specific" methods that can ...
0
votes
0answers
13 views
How can I analyse simmilarities/differences of one nominal variable of two different groups?
I have two different samples. I want to measure how similar is each individual of the first group with the second group in terms of Var 1 (nominal not ordered), given that they are both categorized ...
