Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

Filter by
Sorted by
Tagged with
1
vote
1answer
36 views

Intuition on Wasserstein Distance

I've been trying to familiarize myself with the Wasserstein distance and saw this answer on StackExchange by @antike that at first made a lot of sense, but then it didn't (to me, of course). In the ...
0
votes
0answers
9 views

Using Logistic regression in record linkage

I am curious as to how logistic regression handles string variables in a training matched data set I am aware many use Logistic regression to categorize data that includes the process of matching ...
1
vote
1answer
49 views
+50

Non-logarithmic approaches to compositional data

Background Compositional data ($x_i>0, \sum_i x_i=c$) are usually analyzed using some kind of log-transformation (alr/clr/ilr), to take into account naturally the fact that, in presence of the sum ...
0
votes
1answer
25 views

About $D(F,G)=\int(F(x)-G(x))^2w(x)dx$ and $D(F,G)=\int(F(x)-G(x))^2w(x)dF(x)$

I learned that statistical distance between two 1-dim distributions F and G $D_E(F,G)=\int(F(x)-G(x))^2dx$ is famous. But what about $D(F,G)=\int(F(x)-G(x))^2w(x)dx$ or $D(F,G)=\int(F(x)-G(x))^2w(x)dF(...
0
votes
1answer
32 views

What is the name of $D(F,G)=\int(F(x)-G(x))^2dF(x)$?

Is there a statistical distance between two 1-dim distribution F and G that $D(F,G)=\int(F(x)-G(x))^2dF(x)$? Or to symmetrize it, take $D^s(F,G)=\int(F(x)-G(x))^2dF(x)+dG(x)$ If not, why? (What are ...
0
votes
1answer
35 views

Is it sensible to do PCA on a distance matrix?

I have 10x10 distance matrix where the distance metrics is (1 - overlap coefficient). I want to represent the observations in this matrix in a low dimensional space to see how observations relate to ...
0
votes
0answers
17 views

Significance testing for Jensen–Shannon divergence?

The Jensen-Shannon divergence (JSD) measures the (dis)similarity between multiple probability distributions. How can one determine whether the JSD of (a pair of, or multiple) distributions is ...
1
vote
1answer
26 views

Documentation or code sample for agglomerative clustering

for my undergraduate research project, I'm looking for an R code for agglomerative clustering. Basically, I need to know what happened inside hclust method in R. I have looked everywhere but don't ...
4
votes
1answer
59 views

KL-divergence for joint probability distributions?

I have a pair of joint probability distributions. I want to measure their similarity/dissimilarity. If they were single-dimensional probability distributions, then I could measure the KL-divergence or ...
0
votes
0answers
14 views

Significance test for comparing two mean Euclidean distances?

First time poster here, so please let me know if more details are needed! In short, I'm analyzing a dataset that has scores from 98 siblings from 64 families. So some families contributed only one ...
0
votes
0answers
21 views

How to simultaneously compare individuals in multiple dimensions

I have collected data about subjects in 20 different variables (all of them are continuous and have been standardized as Z scores). What I would like to do now is to obtain a valid measure of the ...
0
votes
1answer
30 views

Expected distance to nearest neighbour: Why integral of survivor function? (instead of derivative)

Previously, I asked the question how to calculate the expected distance to the nearest neighbor molecule in 3-dimensional space. This question was fully answered, which is I ask this related question ...
0
votes
0answers
13 views

Composed pair-wise similarity measure

I'm dealing with a graph theory problem for which I have calculated a series of pair-wise similarity measures (several criteria such as ancestrality, co-occurrence, sentence similarity, etc.) between ...
0
votes
0answers
18 views

Best way to evaluate ranking when one only has pairwise distances?

Let's say I have a strict ranked set of samples. I only have a similarity measure $s$. I want to evaluate how good this similarity measure is at ranking the examples. One approach would be to use a ...
0
votes
0answers
6 views

Patch wise feature vector comparison

I have a image of size of 64*64. I am trying to compute HOG features for the image. I have skimage for my implementation, with the following parameters: ...
0
votes
0answers
14 views

Distance measure for hierarchical nominal data

I have categorical data which follow a hierarchical structure (in fact they're medical codes). For instance: C10: Diabetes Mellitus E00: Senile dementia E10: Schizophrenia E2B1: Chronic Depression G20:...
2
votes
1answer
22 views

Dice Distance returning nan. Workaround?

Starting Point: I want to calculate the distances between nominal data. Furthermore, I want to see what difference it makes to only use important features (given some feature selection method). So I ...
0
votes
0answers
21 views

Quantifying if two datasets are from the same distribution, if I only have distances

Let's say that I have two datasets. The target dataset is 1K instances, the "predicted" dataset is 1M instances (but I could downsample). I can compute the distance between instances. How do ...
0
votes
0answers
27 views

Mantel test alternatives: linear mixed models, with row and column ids of distance matrices as random effects?

Summary: I want to model in R the relationships between pairwise spatial distances, pairwise temporal distances, and pairwise Jaccard distances, with the goal of predicting the Jaccard distance ...
1
vote
0answers
29 views

Distance to uniform distribution for continuous probability distributions [closed]

I need to quantify the distance of a continuous probability distribution $f(x)$ to the uniform distribution on $x \in (-\infty, +\infty)$. Do you know any distance functions that could help here? For ...
3
votes
2answers
160 views

What are the pros and cons of using mahalanobis distance instead of propensity scores in matching

I learned about this option of using mahalanobis distance instead of PS to do matching from the matchit() function in R. It seems a more nonparametric approach. Could you state its pros and cons and ...
0
votes
0answers
21 views

Clustering with booleans and continuous data; Gower's coefficient + PAM?

I have a medical dataset with both boolean variables and continuous variables (e.g. age/BMI). I know that clustering with K-means won't work due to the mixed datatypes. I read that I can use the Gower'...
0
votes
0answers
39 views

Calculate similarity between two matrices

I have two matrices, $A$ and $B$, each of size $n\times m$, where $n$ is discrete time points, and $m$ are the variables measured (specifically, $n$ are dates and $m$ are investments measured in ...
0
votes
0answers
14 views

Options for Clustering Analysis with Numeric & Nominal Data with Gower Distance

I am working through some cluster analysis (trying to propose new item types for various clusters). I have data that has both numeric and nominal features. After creating dummy variables for all ...
0
votes
0answers
7 views

Ranking items by the magnitude of their effect on dissimilarity?

[reposting with more detail, after previous question was removed due to lack of detail or clarity] I am working on getting a better understanding of my company's user base. We have distinct ...
0
votes
0answers
15 views

Measuring the overlap between two probability distribution [duplicate]

I have many probability distributions, I need to compute the amount of overlap between two probability distributions. I don't know the type of distribution since it really depends on the data itself. ...
1
vote
0answers
15 views

Hausdorff Distance with Manhattan Distance

I'm applying Hausdorff Distance to understand if two datasets are representing the same subset of the space of a particular problem. The point is that I've read the Hausdorff distance computes the ...
1
vote
1answer
30 views

How do measure how different two policies are?

I have two agents that both follow a baseline behavioral policy pi(a|s). If I then modify the state-action distribution for the two agents (resulting in two new policies), is there a standard measure ...
0
votes
1answer
21 views

Is Euclidean distance the same as distance-from-correlation as $d(x, y) = \sqrt{2m[1 - r(x, y)]}$

I found in a couple of documents (e.g. this) that the Euclidean distance $d(x, y) = \sqrt{\sum_{i = 1}^{n}{(x_i - y_i)^2}}$ can be obtained from correlation coeffcient if $x$ and $y$ are standardised ...
0
votes
0answers
22 views

Similarity Metric Validation

I want to score a number of similarity metrics, i.e. given a function s(x,y) which returns a number that is higher the more similar x and y are. I'm want to objectively score a number of different ...
0
votes
0answers
31 views

Gaussian Mixture Models and distance matrix

I have a (euclidean) distance matrix and I want to perform GMM clustering. I read in another post (gaussian mixture model - approximate a matrix) that I could apply MDS or PCA to this matrix and use ...
0
votes
0answers
43 views

Wasserstein distance for categorical data? Relationship to TVD?

Is the Wasserstein distance applicable for categorical data? e.g. if we have the distribution of different coloured balls in two bags, ...
0
votes
1answer
26 views

Is the set of distribution $\{ X | \max_t |f_X(t) - f_Y(t)| \leq \epsilon \}$ convex, where f is the cdf or inverse cdf?

I'm trying to figure out if the set is convex, where the maximum difference between cdf(or inverse cdf) of X and a reference distribution Y is smaller than $\epsilon$. 1. Let $f_X(t)$ denote the cdf ...
0
votes
0answers
13 views

i.i.d data for computing distance correlation

When computing an estimate of the distance correlation of random vectors $X$ and $Y$ using paired samples of the vectors, should the samples be i.i.d? Can I have data in which $X_i$ and $X_j$ are ...
2
votes
1answer
30 views

Probability of points less than a fixed distance apart in a vector space

I have a distribution $D$ of points in a normed vector space (it's $\mathbb{R}^n$ using the $L_\infty$ norm, but I don't think that matters). In this particular space, points that are less than a ...
0
votes
0answers
18 views

Computing the most representative sample of a random variable

Let $X$ be a real-valued random variable and $n > 0$. Using numerical methods, how can we find the vector $\vec v$ of $n$ real numbers that is most characteristic of $X$, in the sense that the ...
0
votes
1answer
24 views

Measure of distance between two survey responses

I've found some survey data where respondents answer 63 question by giving a response for each question between 0-10 (0 for strong disagreement, 10 for strong agreement). So I can view every ...
0
votes
1answer
73 views

Similarity/dissimilarity matrix over classes

I am trying to get something like a confusion matrix for different classes, but without training a model. The idea is to use some kind of distance between classes. The data set is like this, just for ...
1
vote
0answers
18 views

J-S Divergence as a Percentage

Is there a way to interpret the Jensen-Shannon divergence which is normalized to be between 0 and 1 between two probability distributions as a "percent difference", i.e., there is a x% ...
1
vote
0answers
15 views

Clustering - Distance Metric for Comparing Short Lists of Terms (non-repeating, no frequency)

Clustering involves using some distance or similarity metric. What is the best way to score the similarity of these small sets of words? Criteria: These are technical terms which are extracted from ...
0
votes
1answer
94 views

Using $\chi^2$-Distance for Histogram-Comparison with 0-Valued Elements leads to NaNs

I want to compare two histograms by using the $\chi^2$-distance. There is a definition in the OpenCV-library. Also this question gives a lot of insight about this distance-metric. Because i didn't ...
0
votes
0answers
29 views

K-Means transform function does not match pairwise_distances_argmin_min centar calculation

I need to be able to select first N most representative points from each cluster calculated by K-means. To do so, I am aiming to calulate the distance of each point to its own cluster center and take ...
0
votes
0answers
22 views

Specific proof related to MDS distance matrix

Given a symmetric, positive semidefinite matrix A, and matrix D, where $D_{ij}=A_{ii}-2A_{ij}+A_{jj}$, prove that there exist n vectors {$\vec{v_1},...,\vec{v_n}$} such that $D_{ij}=||\vec{v_i}-\vec{...
1
vote
0answers
63 views

Product of two probability density function

Suppose $f$ and $g$ are two probability density functions. I have seen economists use $\int f(x)g(x) dx$ as some kind of similarity measure. For example, Jaffe (1986) uses sum of product of two ...
0
votes
0answers
32 views

Distance of a vector to a sample of a distribution?

Is there a better measure than Euclidean distance for my problem? I have two functions e() and f(x). x is a vector of continuous values. The process e() outputs a variable z. Subsequently, a variable ...
1
vote
0answers
26 views

Is there a metric to identify whether samples are uniformly distributed when number of samples is small?

Suppose I have a small number of samples drawn from an unknown distribution $\{X_1,X_2,...,X_n\}$, where $0\le X_i \le L$, and $3\le n \le10$. I want to identify a metric to understand how far these ...
3
votes
0answers
79 views

Finding a sub-population from dataset matching another target dataset

Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
3
votes
1answer
55 views

Distance matrix time series analysis? (Ecology/diversity)

I am trying to analyze a time series of ecological data. Each time point in the series is a matrix of animals-by-foods (that they were observed to eat). For each of these time points, I compute the ...
6
votes
1answer
34 views

two times square in distance calculation on one example?

I read a book on Kernels, See the following example. Why the authors take square two times here? what is the logic?
2
votes
0answers
40 views

Best approximation of the Mahalanobis distance by standardized Euclidean distance

I am looking for the best way to approximate the Mahalanobis distance by the standardized Euclidean distance, which would reduce the number of the required multiplications. The easiest way is the ...

1
2 3 4 5
13