Questions tagged [distance]

Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

Filter by
Sorted by
Tagged with
0
votes
1answer
159 views
0
votes
0answers
3 views

dbRDA and variance partitioning with time series

I'm trying to carry outa distance-based RDA (dbRDA) on some data which comes from repetitive biotic (table of abundances) and abiotic measurements at a single location (i.e. a multivariate time serie)....
0
votes
0answers
10 views

Is the Bray-Curtis dissimilarity/similarity independent of richness/diversity?

I was told about potential issues with the Bray-Curtis dissimilarity/similarity measure. Specifically, I have been asked if Bray-Curtis is independent of richness/diversity. I am unsure how to answer ...
0
votes
0answers
12 views

Distance between stochastic processes controlled by power spectral density

Let $f_1$ and $f_2$ be two stochastic processes over the same domain with finite power spectral densities (PSDs) $S_1$ and $S_2$ respectively. Can I bound a distance between $f_1$ and $f_2$ based on ...
0
votes
0answers
78 views

Interpretation of upper bound on the Wasserstein Distance

I am trying to interpret the 2-Wasserstein distance and the upper bound on it. Let's say I have 2-Wasserstein distance between two distributions to be $x$, and I have an upper bound on it which gives ...
0
votes
0answers
12 views

What distance measure should be used to cluster (A - B) where A and B are correlation matrices?

I have 2 conditions with $m$ variables: Treatment = $X_A$ Reference = $X_B$ I've calculated the pairwise correlation of each condition to have 2 $m x m$ correlation matrices I'm calling: Treatment ...
6
votes
2answers
539 views

Expected squared distance from origin of training points vs. test points

This is from Exercise 2.4 (Page 39) of Elements of Statistical Learning: The edge effect problem discussed on page 23 is not peculiar to uniform sampling from bounded domains. Consider inputs drawn ...
1
vote
1answer
31 views

Distance of a covariance matrix from a perfect 1:1 relationship

I have a number of estimated variance-covariance matrices, and I would like to know how different these are from a perfect 1:1 relationship. To be specific: They are (genetic) variance-covariances ...
0
votes
0answers
15 views

What is the correct distance metric for hour?

I want to cluster points using time and spatial coordinates. My time dimension is only containing hour of day. In order to have a correct representation of time I transform my hour using ...
2
votes
2answers
534 views

Can I compare Mahalanobis distances from different distributions?

I have a multivariate dataset representing multiple locations, each of which has a set of reference observations and a single test observation. For each location, I would like to measure how anomalous ...
0
votes
0answers
7 views

Correlation-based distance dissimilarity measure is not a metric

Show by an example that the correlation-based distance $d(X,X^\prime)=1-\rho(X,X^\prime)=1-\frac{\sum_{j=1}^p (X_j-\bar{X})((X_j^\prime-\bar{X}^\prime)}{\sqrt(\sum_{j=1}^p (X_j-\bar{X})^2\sum_{j=1}^p (...
27
votes
3answers
37k views

Distribution of difference between two normal distributions

I have two probability density functions of normal distributions: $$f_1(x_1 \; | \; \mu_1, \sigma_1) = \frac{1}{\sigma_1\sqrt{2\pi} } \; e^{ -\frac{(x-\mu_1)^2}{2\sigma_1^2} }$$ and $$f_2(x_2 \; | \...
4
votes
1answer
110 views

KL divergence for joint probability distributions?

I have a pair of joint probability distributions. I want to measure their similarity/dissimilarity. If they were single-dimensional probability distributions, then I could measure the Kullback–Leibler ...
0
votes
0answers
102 views

Interpreting exercise in Elements of Statistical Learning

I am reading exercise 6.4 from The Elements of Statistical Learning (Hastie, Tibshirani and Friedman) and I am having difficulty interpreting exactly what is being asked in the following question ...
0
votes
0answers
12 views

How to cluster points spatially using a maximum radius as a constraint?

I am building an app to optimize video packet sharing between users that are watching the same video stream at the same time. I do not want to have to guess the number of clusters up front because I ...
0
votes
0answers
13 views

How to make a Bayesian adaptation of a null hypothesis test?

I am trying to make software to detect anomalies from our instruments. We have a pair of instruments that each measure the same quantity but in a different way. Both instruments report a probability ...
0
votes
0answers
25 views

Reconciling cosine similarity between vectors and subsets of these vectors

I'm seeing something that I'm having a hard time reconciling in my head. Essentially, the cosine similarity between two vectors I have is very low, but cosine similarities of their subsets are very ...
1
vote
1answer
45 views

Intuition on Wasserstein Distance

I've been trying to familiarize myself with the Wasserstein distance and saw this answer on StackExchange by @antike that at first made a lot of sense, but then it didn't (to me, of course). In the ...
0
votes
0answers
15 views

Distance measure for hierarchical nominal data

I have categorical data which follow a hierarchical structure (in fact they're medical codes). For instance: C10: Diabetes Mellitus E00: Senile dementia E10: Schizophrenia E2B1: Chronic Depression G20:...
0
votes
1answer
126 views

Clustering given “distance” matrix and K in python

INPUT ($D$, $K$): I have a symmetrical "distance" matrix $D$ of size $N \times N$ which tells me how distant one object is from another. Function used for calculating the distances is not a ...
1
vote
1answer
64 views

Non-logarithmic approaches to compositional data

Background Compositional data ($x_i>0, \sum_i x_i=c$) are usually analyzed using some kind of log-transformation (alr/clr/ilr), to take into account naturally the fact that, in presence of the sum ...
0
votes
1answer
306 views

Euclidean distances in spaces of different dimensions

A modest attempt to illustrate my question: Setup Consider a survey where you have to choose if you like to do an activity or not. You do this for a number of activities $A_1,...,A_8$. In addition, ...
0
votes
0answers
9 views

Using Logistic regression in record linkage

I am curious as to how logistic regression handles string variables in a training matched data set I am aware many use Logistic regression to categorize data that includes the process of matching ...
1
vote
1answer
759 views

How to extend the dynamic time warping to 3D Cartesian space

I already searched the topics here but couldn't figure out clearly what to do so I'm asking here. I have thousands of fiber tracts (the outputs of a tractography algorithm on brain diffusion MRI) that ...
0
votes
1answer
347 views

Word vector normalisation by document size

I have a bunch of text documents of varying lengths (100k words to just thousands). I want to compare similarities of these vectors, specifically, cosine similarities. While I understand that cosine ...
0
votes
1answer
25 views

About $D(F,G)=\int(F(x)-G(x))^2w(x)dx$ and $D(F,G)=\int(F(x)-G(x))^2w(x)dF(x)$

I learned that statistical distance between two 1-dim distributions F and G $D_E(F,G)=\int(F(x)-G(x))^2dx$ is famous. But what about $D(F,G)=\int(F(x)-G(x))^2w(x)dx$ or $D(F,G)=\int(F(x)-G(x))^2w(x)dF(...
0
votes
1answer
34 views

What is the name of $D(F,G)=\int(F(x)-G(x))^2dF(x)$?

Is there a statistical distance between two 1-dim distribution F and G that $D(F,G)=\int(F(x)-G(x))^2dF(x)$? Or to symmetrize it, take $D^s(F,G)=\int(F(x)-G(x))^2dF(x)+dG(x)$ If not, why? (What are ...
0
votes
1answer
47 views

Is it sensible to do PCA on a distance matrix?

I have 10x10 distance matrix where the distance metrics is (1 - overlap coefficient). I want to represent the observations in this matrix in a low dimensional space to see how observations relate to ...
0
votes
0answers
20 views

Significance testing for Jensen–Shannon divergence?

The Jensen-Shannon divergence (JSD) measures the (dis)similarity between multiple probability distributions. How can one determine whether the JSD of (a pair of, or multiple) distributions is ...
4
votes
1answer
173 views

Is random walk a good metric to compute distances of two sets of nodes in the graph?

I have a big graph and I have three sets(A,B,C) of labeled nodes on it. I would like to compute on average how fare each sets are from each others. In other word, I want to compute distance matrix for ...
0
votes
0answers
14 views

Significance test for comparing two mean Euclidean distances?

First time poster here, so please let me know if more details are needed! In short, I'm analyzing a dataset that has scores from 98 siblings from 64 families. So some families contributed only one ...
0
votes
1answer
31 views

Expected distance to nearest neighbour: Why integral of survivor function? (instead of derivative)

Previously, I asked the question how to calculate the expected distance to the nearest neighbor molecule in 3-dimensional space. This question was fully answered, which is I ask this related question ...
0
votes
0answers
14 views

Composed pair-wise similarity measure

I'm dealing with a graph theory problem for which I have calculated a series of pair-wise similarity measures (several criteria such as ancestrality, co-occurrence, sentence similarity, etc.) between ...
13
votes
4answers
6k views

Is triangle inequality fulfilled for these correlation-based distances?

For hierarchical clustering I often see the following two "metrics" (they aren't exactly speaking) for measuring the distance between two random variables $X$ and $Y$: $\newcommand{\Cor}{\mathrm{Cor}}$...
0
votes
0answers
18 views

Best way to evaluate ranking when one only has pairwise distances?

Let's say I have a strict ranked set of samples. I only have a similarity measure $s$. I want to evaluate how good this similarity measure is at ranking the examples. One approach would be to use a ...
0
votes
0answers
6 views

Patch wise feature vector comparison

I have a image of size of 64*64. I am trying to compute HOG features for the image. I have skimage for my implementation, with the following parameters: ...
13
votes
3answers
35k views

Which distance to use? e.g., manhattan, euclidean, Bray-Curtis, etc

I am not a community ecologist, but these days I am working on community ecology data. What I couldn't understand, apart from the mathematics of these distances, is the criteria for each distance to ...
2
votes
1answer
23 views

Dice Distance returning nan. Workaround?

Starting Point: I want to calculate the distances between nominal data. Furthermore, I want to see what difference it makes to only use important features (given some feature selection method). So I ...
2
votes
0answers
43 views

Detection Function in Distance Sampling, R

I'm fitting detection functions on whale data using the ds function in Distance package in R. I am having difficulty choosing the best detection function to then use in a density surface model. I ...
0
votes
0answers
23 views

Quantifying if two datasets are from the same distribution, if I only have distances

Let's say that I have two datasets. The target dataset is 1K instances, the "predicted" dataset is 1M instances (but I could downsample). I can compute the distance between instances. How do ...
0
votes
0answers
32 views

Mantel test alternatives: linear mixed models, with row and column ids of distance matrices as random effects?

Summary: I want to model in R the relationships between pairwise spatial distances, pairwise temporal distances, and pairwise Jaccard distances, with the goal of predicting the Jaccard distance ...
1
vote
0answers
29 views

Distance to uniform distribution for continuous probability distributions [closed]

I need to quantify the distance of a continuous probability distribution $f(x)$ to the uniform distribution on $x \in (-\infty, +\infty)$. Do you know any distance functions that could help here? For ...
1
vote
1answer
191 views

Odd-one-out survey

I'm currently conducting a survey on a pool of songs, divided in rounds. For each round, the user listens to 3 songs, and then choses the one he thinks is the odd-one-out (similarity-wise). I'm ...
3
votes
2answers
213 views

What are the pros and cons of using mahalanobis distance instead of propensity scores in matching

I learned about this option of using mahalanobis distance instead of PS to do matching from the matchit() function in R. It seems a more nonparametric approach. Could you state its pros and cons and ...
15
votes
4answers
33k views

Definition of normalized Euclidean distance

Recently I have started looking for the definition of normalized Euclidean distance between two real vectors $u$ and $v$. So far, I have discovered two apparently unrelated definitions: http://en....
10
votes
1answer
2k views

Efficient/fast Mahalanobis distance computation

Suppose I have $n$ data points $x_1,\dots,x_n$, each of which is $p$-dimensional. Let $\Sigma$ be the (non-singular) population covariance of these samples. With respect to $\Sigma$, what is the most ...
0
votes
0answers
21 views

Clustering with booleans and continuous data; Gower's coefficient + PAM?

I have a medical dataset with both boolean variables and continuous variables (e.g. age/BMI). I know that clustering with K-means won't work due to the mixed datatypes. I read that I can use the Gower'...
1
vote
1answer
149 views

Something like Mahalanobis distance when the copula is not Gaussian

Mahalanobis distance accounts for different variances of the marginal variables and correlations between the marginal variables. However, there is an implicit (maybe explicit) assumption that ...
0
votes
0answers
101 views

Calculate similarity between two matrices

I have two matrices, $A$ and $B$, each of size $n\times m$, where $n$ is discrete time points, and $m$ are the variables measured (specifically, $n$ are dates and $m$ are investments measured in ...
0
votes
0answers
29 views

Options for Clustering Analysis with Numeric & Nominal Data with Gower Distance

I am working through some cluster analysis (trying to propose new item types for various clusters). I have data that has both numeric and nominal features. After creating dummy variables for all ...

1
2 3 4 5
14