Questions tagged [distance]
Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.
0
votes
1answer
30 views
R: What is the distance function equivalent for this formula?
Hi I'm using an R package that calculates distance with this formula here,
as.dist(1 - cor(df, use = "pa"))
However I cannot seem to find an equivalent dist ...
0
votes
0answers
18 views
How to localize points from an incomplete distance matrix in R?
Suppose you have 3 shops and 2 supply units, and you only know the 6 pairwise (Euclidean, assuming 2D) distances between each shop and each supply unit, but not the pairwise distances between the ...
0
votes
0answers
3 views
Quick search based on similarity: logarithmic time
I have some objects $x\in X$ and a metric $s:X\times X\to\mathbb{R_{+}}$. For each $x$, there is a $y\in Y$. Note that $x$ and $y$ are highly structured and we cannot consider neural networks for ...
0
votes
0answers
7 views
Calculate association trends
I have time series dataset for 10 consecutive periods (i.e. T, T+1, T+2, ..., T+9). Moreover, I also have 100 term triplets in each time period. Each triplet contains 3 objects namely x, y and z.
I ...
3
votes
1answer
40 views
Calculation of relative distance
I have 100 term triplets as shown in the below mentioned figure. Each triplet contains 3 objects namely x, y and z. I want to rank the triplets according to the following two properties.
y should be ...
1
vote
1answer
52 views
Mahalanobis distance gives counterintuitive results [closed]
I have generated 100 sample time series, each 24 items long, and each with an exponential distribution with a different scale for each of the 24 time points. This is the scale parameter per time point:...
0
votes
0answers
33 views
Can Mahalanobis distance not be applied reliably after performing one-hot encoding with certain data-sets?
I am working with a data-set of patient performance-data and patient demographics for people with a medical condition. I am trying to assess the effect of a treatment on the patients, and as I am ...
0
votes
0answers
11 views
Variable transformation for converting Euclidean distance to Mahalanobis distance
I'm working on a project that involves computing distances for correlated variables with dissimilar variances for approximate Bayesian computation. Sounds like a job for Mahalanobis distances. ...
2
votes
1answer
45 views
Basic Hellinger Distance
One definition of Hellinger Distance is $$L_{H}: (1/2) E_{\theta}([\sqrt{\frac{f(x|\delta)}{f(x|\theta)}}-1])^{2}$$
My book has that for $x \sim N(\theta,1)$
$$L_{H}(\theta,\delta)=1-\exp(-(1/8)(\...
1
vote
0answers
30 views
Mahalanobis distance between 2 points doesn't work when covariance matrix has values close to 0
I am working on a project where I am trying to replicate a randomized experiment from an observational study data, using Mahalanobis distance matching to ensure that the control and treated groups are ...
2
votes
1answer
40 views
How can I order kmeans clusters?
I have a kmeans cluster object and I would like to order the clusters. Not the observations within the clusters, rather the clusters in order of each other.
Is there a way of doing this? I found ...
0
votes
1answer
20 views
Kmeans results, is the cluster vector ordered by 'closeness"?
I ran kmeans in r with k = 20 centers and 7 scaled variables to cluster with on a data frame with n = 100K.
Using dplyr group_by I was able to view summary data for each of the 20 clusters: the mean ...
-1
votes
1answer
27 views
How k-means computes cluster centroids differently for each distance metric?
K-means computes cluster centroids differently for each distance metric. I don't know why the way of computing the centroid is dependent of the distance measure.
I don't know how we compute the ...
0
votes
0answers
12 views
Robust distance measure for correlated data
I read a paper in which the authors want to compare the overall predictive accuracy of various predictors on a set of variables by using the Mahalanobis-Distance. However the data is not even ...
0
votes
0answers
18 views
what is the mathematical difference between the distances when clustering text
Suppose i have data for text clustering, there 300 000 rows text$GOODS_NAME
let's do text clustering.
...
0
votes
0answers
12 views
Methods to minimize distance between features
What are some methods to minimize distances between features belonging to a cluster? I want to do this regardless of their class. Say I have a $m\times d$ feature matrix of $m$ samples, each with $d$ ...
3
votes
2answers
131 views
Correcting Kullback-Leibler divergence for size of datasets
We have the following implementation of KLD:
...
0
votes
1answer
20 views
Clustering by same random projection
I have $N$, $1024$-dimensional vectors. I want to cluster them by some similarity. Given the high dimensionality, standard metrics won't work. I tried a few Approximate Nearest Neighbor ...
1
vote
0answers
36 views
Compare KS test and Wasserstein distance or Earth mover's distance
Consider two sets of data points A and B. Both these data points are from mixture of unknown number of Gaussians. The mean of the Gaussians are little different for each set (there may have few ...
0
votes
0answers
14 views
Comparison of empirical discrete distributions. Pros and cons of different metrics?
I am trying to measure the dissimilarity between two empirical discrete distributions. I am aware of various distance metrics that could be used for this purpose such as Wasserstein, Bhattacharyya etc....
1
vote
0answers
15 views
Mean distance from the centre to any point in a sphere and a cylinder [closed]
What is the mean distance from the centre to any point within a sphere of radius r? What is the mean distance from the centre to any point within a cylinder of radius r and length l?
0
votes
0answers
37 views
How to quantify distance between 2 datasets?
I have a distribution $A$ (intent-to-treat population) and its subset $B \subset A$ (treated population).
I learn a propensity model $P(x \in B)$ to predict treatment.
Then I sample the intent-to-...
1
vote
0answers
48 views
Distance between angle distributions
I want to quantify the complexity of the street network of different cities.
For each city I have the angle distribution of its streets.
My hypothesis that the more complex the street network, the ...
0
votes
0answers
38 views
Distance between time series and representation (Piecewise Aggregate Approximation - PAA) (R)
I am trying to come up with a way of measuring the distance between a time series and its representation in order to see how closely the representation describes the original series. I feel like there ...
0
votes
0answers
22 views
How to find Mahalanobis distance between an observation and a population with mixed data?
I have a dataset(let's call it as 'D') with multiple continuous, nominal and ordinal variables, as follows:
continuous: Total sales as well as sales figures of some products by customers
nominal: ...
0
votes
0answers
26 views
Comparing distances evaluated on different vector spaces
We have a dataset of I items who have been measured over two different sets of features A, with cardinality N, and B with cardinality M, and N > M. We would like to know in which feature space the ...
3
votes
3answers
186 views
Probability that an observation comes from population A or B?
I'm a web developer looking into some basic statistics -- pardon me if I am using the wrong jargon. :)
Considering that:
I have 2 populations (A and B; each have about 10,000 observations)
For each ...
0
votes
1answer
40 views
Hellinger distance for two shifted log-normal distributions
If I am not mistaken, Hellinger distance between P and Q is generally given by:
$$
H^2(P, Q) = \frac12 \int \left( \sqrt{dP} - \sqrt{dQ} \right)^2
.$$
If P and Q, however, are two differently ...
0
votes
0answers
12 views
How to Measure Similarity to Ground Truth? (Measuring Similarity with Features of Different Scales/Units)
I am trying to reproduce a set of ground truth data
[t_start | t_end | theta_start | theta_end] (blue)
A plot of my ground truth data would show a scatter of lines along the time and angular axes....
2
votes
1answer
60 views
Choosing appropriate distance metric and algorithm for clustering for any given dataset
I have been looking for an answer/guidance/pointer to this question of mine for a while. After going through many (100s actually) posts and articles, I finally found this question, where this response ...
2
votes
1answer
41 views
Distribution of distance of N-1 gamma distributed iid random variables from minimum
I have the minimum value of N iid random variables that are gamma-distributed. The parameters of the gamma distribution are known. What would be the distribution of the distance of the remaining N - 1 ...
0
votes
1answer
26 views
detect an outlier in multi-dimensions - where the number of rows is not >> number of columns
I have the following situation: a data frame with two dimensions x and y, with three "areas": an a-parametric distribution ...
1
vote
0answers
63 views
Is pairwise distance matrix useful to k-means?
The k-means implemented in scikit-learn precomputes distances but I don't how these distances are used. In its standard version, k-means is known to compute only the distances between the points and ...
5
votes
1answer
270 views
Suitable distance metric for time-series clustering with respect to location of shapes
I'm doing clustering on time-series (each time-series has the information for one day = 24 hours). For the clustering purpose, it's important for me to consider the time period in which the shape of ...
2
votes
0answers
127 views
Obtain within-group Gram matrix out of distance matrix
Gram matrix
Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix ...
0
votes
0answers
31 views
How to compute distances with both categorical and continuous attributes?
I have to handle with a datast containing both categorical attributes (around 25) and continuous attributes (around 25). I would like to do outliers detection. I think that it would be a good idea to ...
0
votes
1answer
16 views
Single Linkage Clustering with Manhattan metric
Say suppose we are having 5 data points with 3 attributes each ...
(4,3,1) (2,1,5) (1,2,3) (2,3,1) ....
Now let us build the distance matrix. If we do Manhattan metric then the cell corresponding to ...
1
vote
0answers
61 views
What happens with Mahalanobis-Distance, when the assumption of equal Covariance-Matrices breaks down
Assume that we want to compare the forecast quality of various forecasters $f$ on $n$ values such as stock-market prices or whatever. We could then define a "Mahalanobis-Distance" (MD) (or rather ...
0
votes
0answers
30 views
Common methods to calculate total distance from data with categorical, continuous and counting variables
I have a data set with categorical, continuous and counting variables. I want to be able to use a method that will give me a distance for each pairwise data point.
From my understanding, each type ...
0
votes
0answers
40 views
Similarity of two sets of points
PROBLEM
I have one set of 10 points, $X = \{(x_1,y_1),\,\dots,\,(x_{10},y_{10})\}$
and two sets of 3 points each, $A = \{ (a_1,b_1),\, (a_2,b_2),\, (a_3,b_3) \}$ and $C = \{(c_1,d_1),\,(c_2,d_2),\,(...
0
votes
0answers
18 views
Which is the best distance metric in an Indicator matrix
Is it okay to use the $\chi^2-distance$ when we have a indicator matrix? With Indicator matrix I mean the complete dijuntive table that is used in the Multiple Correspondence analysis.
I mea
n, if we ...
0
votes
1answer
83 views
Is Earth Mover Distance has maximum bound?
I have two probability distributions which each distribution has sum up to 1.
I want to compute the distance between those two probability distributions. I want to use Earth Mover Distance to ...
4
votes
2answers
110 views
The Curse of high Dimension And Distance
For extracting features from video frames (2 sample/sec) I use keras framework in python and load VGG16 that input size is (150,150,3) and output size is (4,4,512). After the feature extraction step I ...
4
votes
2answers
104 views
Is the maximum bound of Euclidean distance between two probability distributions equal to $\sqrt{2}$?
I used Euclidean distance to compute the distance between two probability distribution. The example of computation shown in the Figure below.
As my understanding, the maximum distance occur while $...
11
votes
3answers
2k views
What's the maximum value of Kullback-Leibler (KL) divergence
I am going to use KL divergence in my python code and I got this tutorial.
On that tutorial, to implement KL divergence is quite simple.
...
6
votes
2answers
297 views
Do Autoencoders preserve distances?
Based on my understanding, autoencoders are used to find a compact representation of input features that carries the essential underlying information.
Is there any relationship between the L2 ...
1
vote
0answers
85 views
Gower's Distance + PCA : How to interpret in R? [closed]
I have a dataset (40 records and 10 variables that is a combination of continuous variables and categorical variables (age, work experience, current salary, educational qualification, house in rural/...
0
votes
0answers
56 views
On the right dissimilarity matrix for discrete cluster analysis
Currently I'm having some trouble on deciding the right dissimilarity matrix for my data. After reading some previously asked questions here, I'm even left more confused. So, here is the deal. My data ...
0
votes
0answers
17 views
How well does a subsample of a large dataset represents the full set in a statistical sense?
Suppose that a large sample $ \{X_k\}_{k=1}^{n} $ (from a multivariate distribution $ X $) is given. I would like find a subsample $ \{ Y_k \}_{k=1}^{m} \subset \{X_k\}_{k=1}^{n} $ of this dataset in ...
1
vote
0answers
20 views
Distance vs Dissimilarity measure
I am reading up on distance and dissimilarity measures for my class on natural language processing and could not understand this slide. Why does the dissimilarity measure not satisfy item 3 ? What ...
