Skip to main content

Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

GLMM with cumulative distances

I have calculated the cumulative distances travelled for each individual. The objective is to represent the data in a graph with a tendency curve using the R package ggplot2. However, the data are too ...
Klervi's user avatar
  • 11
2 votes
1 answer
32 views

analytical asymptotic approximation of the expected maximum, mean, and minimum distance of nearest neighbours in unit ball

Say I uniformly at random distribute $x = n^3$ (independent identically distributed) points in a ball of radius $r=1$ in $\mathbb{R}^3$. What can be said about the expected maximum, minimum, and mean ...
kram1032's user avatar
  • 265
0 votes
0 answers
28 views

Wasserstein distance to assess the degree of normality

The Wasserstein distance between two probability measures with quantile functions $F^{-1}$ and $G^{-1}$ is given by \begin{align} W(F,G) = \int_{[0,1]} |F^{-1}(t) - G^{-1}(t)|dt \end{align} Now let's ...
thesecond's user avatar
  • 390
0 votes
0 answers
21 views

Looking for a suitable way to find groups of events

I have an excel file in which I have three columns. The first one is the name of an event, the second one is the moment the event starts and the third one is the time at which an event ends. Let's say ...
slow_learner's user avatar
1 vote
1 answer
24 views

After matching: How do I interpret the value of the type ‘distance’ (=Propensity score) in the balance measures table of the r-package cobalt bal.tab?

I have used the R-package ‘MatchIt’ to perform (1) a nearest neighbour propensity score matching (NNM) based on the Framingham Heart Study and (2) for comparison, an optimal PS matching (OM) for the ...
user19939387's user avatar
1 vote
1 answer
59 views

Hypothesis testing by asymptotic distribution

Consider the following hypothesis testing problem: under $H_0$: $(X_1,\cdots,X_n) \sim P_n,$ under $H_1$: $(X_1,\cdots,X_n) \sim Q_n.$ We want to show that the minimum testing error goes to zero when $...
efsdfmo12's user avatar
  • 123
1 vote
1 answer
33 views

Distance between two multivariate distributions

I have data structured as followed: 2 years -> several metrics per year (let's say 2000) -> several measurements per metric (let's say 1000 for year 1 and 800 for year 2). So I have a 2000x1000 ...
Lou's user avatar
  • 21
1 vote
1 answer
45 views

Hierarchical clustering of a distance matrix with element weights

I am computing a hierarchical clustering of some geospatial data. I need to add in an element weighting to the approach. My current approach is: I compute temporal cross-correlations between my N ...
JoshD's user avatar
  • 51
2 votes
1 answer
44 views

What are downsides to "genetic matching," particularly outside of causal inference settings?

Multivariate matching methods typically involve two steps. First the user computes $D$, a matrix of the multivariate distances between units. Second, the user applies a matching function (e.g., 1:1 ...
socialscientist's user avatar
1 vote
0 answers
71 views

Understanding $\chi^2$ in plain language [closed]

Suppose I was trying to explain what $\chi^2$ is and why it's important to my grandma. I want to give core intuition to this formula: $$\chi^2 = \sum_{i}\frac{(O_i-E_i)^2}{E_i}$$ I would tell her ...
Swike's user avatar
  • 139
1 vote
0 answers
48 views

Why not use the $L^2$ norm as the difference between two probability distributions (as opposed to KL-Divergence and others) [closed]

So I was wondering why not just use: $$dist(p,q)=\bigg(\int_{x \in X} |p(x)-q(x)|^2 dx\bigg)^{1/2}$$ instead of the commonly used KL-Divergence, which isn't even a distance measure and therefore not ...
Anon's user avatar
  • 121
0 votes
0 answers
10 views

Distance to find similar samples in a multivariate dataset

Apart from Euclidean and Mahalanobis distance metrics. Given a sample with multivariable values, is there a way to find the samples that are similar to the given sample? Does KNN clustering find the ...
sveer's user avatar
  • 103
0 votes
0 answers
85 views

PCA and Gower's Distance

I have a dataset with nutrient information about different ingredients. There are a total of 70 nutrients (numeric features) and 3 categorical features for a total of around 550 ingredients. I am ...
MSingh's user avatar
  • 1
0 votes
0 answers
24 views

Is it possible to convert a similarity (distance) index to correlation coefficient?

I have two cases that each one has some values on a series of variables (e.g. A, B & C). Is it possible to calculate a distance or similarity index between these two cases and then convert it to a ...
Mahdi Karvandi's user avatar
0 votes
0 answers
18 views

Distance metric for dummy and continous variables

I'm trying to apply the KNN regression model to the data I have at my disposal which contains one dummy variable and two continuous variables (which I have normalized). I was wondering if it is okay ...
soph's user avatar
  • 1
0 votes
0 answers
67 views

Measuring the Distance Between KDE Distributions with Different Bin Counts

I have two KDE distributions, each with a different number of bins. I'd like to compare them effectively, and I'm wondering if there's a recommended technique for this. Should I unify the number of ...
Adham Enaya's user avatar
1 vote
1 answer
43 views

Mathematics behind standardizing the data points in machine learning algorithms (e.g., K-means clustering)

For K-means algorithm, among other methods using distance-based measurements to determine similarity between data points, why we have to standardize the data points with mean as 0 and standard ...
Sophia's user avatar
  • 121
1 vote
0 answers
46 views

How do you test if two discrete ECDFs are drawn from the same population?

Background I have two Empirical Cumulative Distribution Functions (ECDFs) based on two samples of very different sizes. Sample 1: 1020 data points, Power-Law-like distribution, discrete data in the ...
Connor's user avatar
  • 655
0 votes
0 answers
63 views

The meaning of probability density functions' product followed by an integration

Scipy's KDE object allows integration of a function multiplied by another KDE object. I assume that this is meant to be used for the estimation of distance between two distributions. As far as I ...
Gideon Kogan's user avatar
0 votes
0 answers
48 views

Earth Mover's Distance (EMD) between two distributions of different dimensionality

When $x_1\sim N(\mu_1, \Sigma_1)$ and $x_2 \sim N(\mu_2, \Sigma_2)$, and $\mu_1$ and $\mu_2$ are both $D$-dimensional, the EMD is well defined and there exists a closed-form solution for it. Is there ...
user1767774's user avatar
0 votes
0 answers
28 views

What are distances, metrics, or divergences to determine if one sample comes from components of a distribution

I have a distribution that, wlog, can be defined as a mixture of component distributions $\mathscr{D} = \alpha_{1}\mathscr{D}_{1} + \alpha_{2}\mathscr{D}_{2} + \dots$. Is there a metric, distance, or ...
Alex Hagen's user avatar
0 votes
0 answers
16 views

Would ratio of intragroup distances make sense for comparing two distributions?

Given two distributions a and b, we can assess the distance between them using e.g. the Wasserstein distance, Energy distance or ...
gebbissimo's user avatar
0 votes
0 answers
45 views

Drawing a dendrogram based on a confusion matrix

I have a simple 2D confusion matrix that I normalised for every true class. Now, what would be the best method to visualise this as a dendrogram? I have seen hierarchical agglomerative clustering on ...
Damiaan Reijnaers's user avatar
7 votes
2 answers
132 views

Designing an experiment to assess the form of a distance measure

BACKGROUND: I do research in budget-proposal aggregation: each person reports their ideal budget-allocation, and a central authority has to decide on an actual budget-allocation. An important ...
Erel Segal-Halevi's user avatar
0 votes
0 answers
28 views

Panel data clustering - how to assess the distance between individuals when the data are multivariate and longitudinal?

I have an (unbalanced) panel dataset with 20 countries, 57 years, and 8 variables, and I would like to cluster the countries according to their dynamic trend in these variables (whether using kmeans ...
last_resource's user avatar
0 votes
0 answers
19 views

Which dissimilarity index to use with categorical ecological data [duplicate]

I am currently working on data representing the abundance of microorganisms in a categorical way, like 0 = no organisms; 1 = 1-5 organisms; 2 = 6-10 and so on (5 being the highest number). And i am ...
user avatar
3 votes
1 answer
128 views

Appropriate distance measure for comparing probability distributions

Suppose I want to compare some countries to see how similar they are with respect to the relative sizes of the industries in them. So I find data on the distribution of GDP across all industries for ...
user32038's user avatar
2 votes
1 answer
44 views

cartesian distance (2-norm) between paired sampled datasets

I have uniformly sampled datasets from two acquisition circuits: an old model which has become difficult (and expensive) to procure parts to continue manufacturing, and a new version using modern ...
Ben Voigt's user avatar
  • 200
0 votes
0 answers
21 views

Are there any research papers which show why Wasserstein distance is better than Jensen-Shannon/KL_div/Bhattacharya distance for specific use cases?

I am trying to find reliable research work which show why displacement based metrics such as Wasserstein distance is a better suited metric than Jensen-Shannon distance in specific use cases and for ...
user17420392's user avatar
1 vote
1 answer
102 views

Gaussian Mixture Model with Minkowski distance

Gaussian Mixture Models assume Mahalanobis distance (essentially L2). Is it possible to use Lp distance in a GMM? Intuitively, in 1-space, distance is clear. In 2-space, the relation between the two ...
olivarb's user avatar
  • 173
0 votes
0 answers
8 views

Approaches to analyse microbiome of closely related individuals

I am currently analysing the gut microbiome of 36 birds under a controlled living environment and diet. Some of them have a particular disease, therefore I'd like to analyse whether the microbiome ...
Blossom's user avatar
0 votes
0 answers
47 views

What would be the most appropriate distance metric for percentage/ratio data?

I have a matrix where each row is an observation (i), each column is a feature (j), and each value is the ratio the feature j is complete in observation i. That is, the values are floats that range ...
O.rka's user avatar
  • 1,472
0 votes
0 answers
44 views

Quantitative Methods for Evaluating Differences Between Two Distributions

I am working with a substantial dataset in which I need to compare the distributions of certain common features across different categories. The challenge I face is that due to the imprecision in ...
Song Nyanko's user avatar
3 votes
1 answer
158 views

Is L1(a,b) <= L1(c, d) if and only if L2(a,b) <= L2(c, d)? L1 vs L2

I am doing machine learning and in one stage, I have to measure tensor vector differences to find the minimum distance. Lets say I have a set of linear tensors with dimension 100. I want to find a ...
AliM's user avatar
  • 131
0 votes
0 answers
21 views

Distance between two distributions with uncertainty / measurement error

I have two empirical distributions $X$ and $Y$, both with the same number of samples (a few thousand). $X$ are true values, they are accurate (i.e. no uncertainty). Values of $Y$, on the other hand, ...
qalis's user avatar
  • 238
6 votes
2 answers
162 views

Bounding the distance of empirical average from its expected value

Suppose we have three sequences of random variables, $(Y_n)_n$, $(W_n)_n$, and $(X_n)_n$ such that: If $Y_n=a$, then $X_n=b$. If $X_n=b$, then $W_n=c$. That is $$ 1_{[Y_n=a]}\leq 1_{[X_n=b]}\leq 1_{[...
Star's user avatar
  • 889
0 votes
0 answers
14 views

Approximating a bivariate distribution with another distribution, which method to use?

Let $X \sim F(;\theta)$ and $Y \sim G(;\eta)$ be two independent continuous random variables. The greek symbols represent the parameters of those distributions. I can easily sample from these ...
Coolio's user avatar
  • 1
0 votes
0 answers
202 views

How can to visualize/plot correlation matrix as a distance matrix of points in space?

It seems to me that the various options for visualizing the correlation matrix in R are quite unintuitive for laymen. They focus on the graphical representation of the correlation matrix as different ...
kwadratens's user avatar
0 votes
0 answers
79 views

Closed form formula for distance of multinomial sample from underlying distribution

Suppose that I have a probability vector $p$ e.g. of size 10, and that I draw a multinomial sample of size $n$ from $p$. Does there exist a closed form formula to compute the expected total variation ...
Caldym's user avatar
  • 1
0 votes
1 answer
70 views

How to compare two multivariate distribution (of distances) to zero in terms of mean and variance in R?

We have N 3D coordinates estimated with two methods and want to compare them with a reference set of N 3D coordinates which is the ground truth, so in notations: ...
michael's user avatar
  • 303
0 votes
0 answers
18 views

Criteria for a Distance Metric to be Compatible with K-means Clustering

Referring to this post, it's mentioned that K-means clustering doesn't inherently rely on the pairwise distances between data points, and not every distance metric is suitable for k-means clustering. ...
Peyman's user avatar
  • 309
2 votes
1 answer
301 views

Normalizing Euclidean distance by the length of the vectors [closed]

Suppose I have 4 vectors, the first 2 vectors are of length 4 and the last 2 vectors are of length 400. all values in the vectors range from 0.5 to 0.6. The Euclidean distance between the last 2 ...
user17420392's user avatar
-1 votes
1 answer
57 views

Apply divergences between two "relative frequency distributions", instead of between two "probability distributions"

Introduction. Recalling that: The frequency is the number of observations of a specific outcome. The relative frequency is a proportion of all observations (frequency / total observations). A "...
Ommo's user avatar
  • 280
0 votes
1 answer
293 views

Statistical test or distance metric for comparing different distributions?

I want to compare the distributions of two different groups to determine if they are statistically different. Here is an example: ...
ALESSANDRO GNUTTI's user avatar
10 votes
3 answers
2k views

How can this counterintiutive result with the Mahalanobis distance be explained?

I encountered a strange issue when performing Mahalanobis distance matching. Let's say I have one treated unit with the following values on two variables: $T:(17, 4)$. I have two control units with ...
Noah's user avatar
  • 35k
1 vote
1 answer
256 views

Relations between the energy distance and MMD

I was wondering if there's any relation between the two metrics. Both measure the distance between distributions (or samples of them). And they seem quite similar. The energy distance can be ...
Maverick Meerkat's user avatar
0 votes
0 answers
88 views

Hausdorff and IoU are stopped changing while dice metric is decreasing

I am facing a problem in Hausdorff and IoU, where they stop learning when reaching a specific value! While the loss and dice metric keeps changing. surface_distance also has a problem since it is ...
AMAS AL's user avatar
  • 33
2 votes
1 answer
67 views

Minimizing a distance metric where all dimensions are small

I'm minimizing distances between two 6 dimensional vectors. I have been using manhattan distance so far and it works ok but my problem would benefit from discriminating between the following two cases ...
arep0's user avatar
  • 23
0 votes
0 answers
38 views

How to quantify the dissimilarity across different types of variables?

I have two dataframes with the same columns but with varying sample sizes. I want to compare corresponding columns for homogeneity (i.e., do they come from the same distribution?). There are different ...
Glue's user avatar
  • 485
1 vote
1 answer
79 views

Upper bound on Kolmogorov-Smirnov distance after some transformation $h$

Problem setup: Suppose $X_1, \ldots, X_n$ is an i.i.d. sample from $F_X$ (CDF), and $Y_1, \ldots, Y_n$ is another i.i.d. sample from $F_Y$ (also CDF). In addition, $h(z_1, \ldots, z_n)$ is a real-...
Ethan's user avatar
  • 43

1
2 3 4 5
15