Distance functions refer to functions used for quantifying the notion of distance between members of a set, or between objects.

learn more… | top users | synonyms

1
vote
0answers
12 views

Bhattacharyya Distance for Age/Gender Groups?

I'm calculating distances for groups based on Age/Gender Compositions (to rank their similarity in demographic composition.) I'm working with the following: Men 18-34, Men 35-49, Men 50-64, Men 65+, ...
1
vote
0answers
17 views

Clustering before or after ordination

Can someone explain the implications of performing clustering either before or after performing NMDS? I have some ecological data and I am performing a clustering analysis to identify communities of ...
4
votes
2answers
63 views

Distribution of “sample” mahalanobis distances

Let $x_1,\dots,x_n$ be i.i.d. observations from $N_p(0,\Sigma)$. Let $\hat S=\frac1n\sum_{i=1}^n x_ix_i^T$ be the sample covariance of the samples. Recall that the Mahalanobis distance is defined: ...
0
votes
0answers
9 views

Statistical distances and time series of distributions clustering

I am interested in clustering $N$ time series of $T$ 'values' each. These values are distributions (which can be represented by their cumulative distribution functions (cdf), or their probability ...
0
votes
0answers
17 views

Classifying with a distance/error metric

I have a bunch of data of the form (x0, x1, x2) -> y (everything is categorical data), and I want a decision/classification rule for predicting y a metric on y's space (there is no "natural" metric) ...
2
votes
1answer
35 views

A similarity measure with binary data: does this one have a name?

There are many binary similarity measures (e.g. Jaccard, Sorensen, etc), each of them is sensitive to different properties of the compared sets. I would like to use the metric $S=\frac{N_{A\bigcap ...
4
votes
2answers
92 views

Looking for a robust, distribution-free/nonparametric distance between multivariate samples

There are many distance functions for distributions out there, but I'm having a hard time wading through them all to find one that is "distribution-free", or "nonparametric", by which I mean only ...
0
votes
1answer
21 views

Finding vectors with an extreme component

I'm looking for a function that measures if a vector component dominates all the rest. Let $$ \mathbf{v} = [v_1, v_2, \ldots, v_n] $$ and assume that it is L2 normalized, $|\mathbf{v}|_2 = ...
0
votes
0answers
6 views

Looking for a “(numerical) reliability index” for computed Bhattacharyya distances

The Bhattacharyya distance $D_B$ between two distributions $p_1$, $p_2$ is given by $$ D_B(p_1, p_2) = \frac{1}{8} (\mu_1 - \mu_2)^T \Sigma^{-1} (\mu_1 - \mu_2) + \frac{1}{2}\log\frac{\det ...
0
votes
1answer
14 views

Is there a distance algorithm similar to Jaccard distance that handles scalar data?

we have the characteristics and (scalar) values of those characteristics for three (or more) people: ...
0
votes
0answers
16 views

calculating distance among unordered set partitions for k-mean clustering?

I have a dataset for which I construct unordered set partitions for each data point, e.g. {{1,2,3}{4,6}{5}} for one and {{1,3}{2,4,5}{6}} for the next. I would like to perform k-means clustering on ...
0
votes
1answer
35 views

Chosing optimal k and optimal distance-metric for k-means [duplicate]

I have a data-set with roughly 20-dimensions and millions of points which I want to cluster. The goal is to find a set of clusters which: Are as distinct as possible from each other (minimum ...
2
votes
1answer
44 views

Turn a distance measure into a kernel function

I have read here that an easy way to turn a distance function $d$ into a similarity function $s$ is to compute: $s = e^{-\gamma * d}$. I believe that this is also what is done with the RBF kernel. ...
2
votes
0answers
31 views

Calculate the Hamming Distance between the two same datasets

How to Calculate the Hamming Distance between two datasets of same points?Both the data sets look exactly the same. http://postimg.org/image/u11qnsolh/ There are two datasets of same number of ...
0
votes
1answer
29 views

FAMES in case of Dynamic Time Warping

I found this paper Using Pivots to Speed-Up k-Medoids Clustering in which authors explain how to use triangular geometry and cosine law to speed up search of new medoids in case of K-medoids. My ...
1
vote
2answers
44 views

Recommended/estimated number of radial basis functions in RBFN

thank you for taking the time to read my question. I am attempting to make a Radial Basis Function Network to see if a relationship exists between input/output data that I have been collecting. I ...
3
votes
1answer
140 views

What is the difference between graphs/networks? [closed]

Note: read down to below "Question" to find the question. Background: In a previous question I asked how to group what I would call nodes on a network graph based on a connectivity matrix. (link) ...
0
votes
0answers
27 views

Gaussian Mixture Model with Custom Distance Metric

I have some 1D data that I want to cluster using Mixture of Gaussian. However, the data "wraps around" at two extremes. Specifically, I have a list of angles from $-\pi$ to $\pi$ and the data near two ...
0
votes
0answers
45 views

Can I compare Mahalanobis distances from different distributions?

I have a multivariate dataset representing multiple locations, each of which has a set of reference observations and a single test observation. For each location, I would like to measure how anomalous ...
3
votes
2answers
99 views

How to measure similarity of bivariate probability distributions?

I have three different distributions of 2D data: or Now I like to know whether distribution two is more similar to distribution one (2 to 1) than distribution three is to distribution one (3 to ...
0
votes
0answers
32 views

evaluate the similarity between two time series

I have two time series, $\mathcal{T}_1$ and $\mathcal{T}_2$, each time series is of two dimensional. One time series is collected from two sensors (SA, SB), and the other is collected from other ...
1
vote
0answers
99 views

Hierarchical clustering, linkage methods and dynamic time warping

My goal is to cluster time series based on their DTW distance. Therefore I've calculated full distance matrices as input for several clustering algorithms. I first had a look at hierarchical methods, ...
1
vote
0answers
42 views

handling categorical data with a large amount of categories

I have data containing few categorical columns with a huge amount of categories at each (more than 1000 different categories at each column). I have to build a predictive model on this data, using the ...
1
vote
3answers
617 views

Ask for suggestions on clustering methods on a large dataset with mixed types of variables

I need to build segmentation on a large customer dataset with more than 300K records and many variables, including continuous like income and age, ordinal like education level and membership level, ...
2
votes
0answers
18 views

Approximation of objective based on statistical distance

I am a computer science researcher (mostly theoretical) currently in midst of statistics and not able to figure out how to proceed. At an abstract level, I have a hypothesis for an unknown ...
1
vote
0answers
160 views

Comparing two vectors with the same numbers in different positions [closed]

I have two vectors of the same size representing one score for different sites (x and y coordinates) under distinct situations (each vector is one situation). At each vector the scores only change ...
0
votes
0answers
54 views

comparing probability histogram

I have two probability histogram samples. I know there are methods(i.e KS test etc) out there to compare histograms but I am trying to compare through simple sum of absolute difference between these ...
1
vote
1answer
178 views

If Manhattan distance always performs better on a dataset…what does it mean?

I'm analyzing my dataset using kNN. I experimented with various distance functions but Manhattan seems to perform better in terms of lowest RMSE over various values of k. I've read a bit about ...
1
vote
0answers
43 views

Distance metrics in “high” dimensions

Thanks in advance: this is a very nice site. This question has been made more specific in response to a comment. original question ----------------- Reading the various sources concerning distance ...
0
votes
2answers
132 views

Mahalanobis distance measure for clustering

Let's say I have a group of clusters. Would you recommend Mahalanobis distance measure for checking if new arrived data belongs to existing clusters or it is an outlier? Also, would you recommend ...
0
votes
1answer
66 views

Similarity scoring to compare multi-dimensional datasets

I am trying to come up with a mechanism of scoring a set of multidimensional datasets based on a similarity with an ideal dataset. Each dataset will all have the same dimensions along with the ideal. ...
3
votes
1answer
46 views

Can I use k-means with a distance matrix composed of percentages? [duplicate]

I have objects o1, o2,...,on and for each pair I calculate a value that measures the pair's difference. This is a percentage, so for example o1o2 differ by 56%. Now I want to cluster this data. I can ...
1
vote
0answers
139 views

Distance measure for categorical attributes for k-Nearest Neighbor

For my class project, I am working on the Kaggle competition - Don't get kicked The project is to classify test data as good/bad buy for cars. There are 34 features and the data is highly skewed. I ...
0
votes
0answers
16 views

Find input image (ID,passport) in imagesDB based on similarity

I would like to decide if image is exists on DB images (pictures of IDs,passport,Stu. card,etc) I thought of KNN alghorithem that will plot the K closest images. Options for distance metric: 1) sum ...
0
votes
1answer
32 views

Why do two identical feature vectors (distance score 0) get different labels in DBSCAN?

I have two identical feature vectors. They have a distance score of 0. I perform DBSCAN Clustering (using sci-kit) and they get different labels. Is this expected behaviour?
0
votes
1answer
54 views

Multivariate nonparametric divergence (or distance) between distributions

For example, we could say I have two fruit classes (oranges and apples) and for each one I measured different statistics of interest, for example: width, height, sugar, water... of a lot of fruit ...
0
votes
0answers
27 views

Selecting kernel or binary similarity measures

Currently, I am facing a choice of encoding some information either in a binary vector or a normalized (Gaussian) floating point vector of the same length. For instance it could be in the format $[ 1, ...
2
votes
0answers
23 views

Kernel Methods for Binary Vectors

I am currently involved in a project which requires a minor point in choosing a proper similarity metric for a set of binary vectors, i.e. all components are either 1 or 0. Currently, the go-to ...
1
vote
1answer
155 views

Why is k-medians typically used with Manhattan rather than Euclidean distance?

K-medians is typically used with Manhattan distance rather than Euclidean distance. Why is this?
1
vote
1answer
120 views

Calculate PCoA scores for dataframe “x”, based on the distance matrix of dataframe “y”

I'm trying to use multivariate techniques to compare two datasets (same structure) that were collected using different sampling techniques. I'd like to compute a PCoA for the first dataset (D1), and ...
0
votes
1answer
17 views

Calculating distance comparing sets of frequencies

I have two sets of items, say A (with items a1, a2..) and B (with items b1,b2..). Each item in A appears with different frequency with items in B, so each item would have a list of B items with ...
0
votes
0answers
17 views

How to split a class which is not very cohesive?

Using the silhouette width metric I can find out as to how well each object lies within its class after classification is done. I next find the average silhouette width of objects within a class and ...
2
votes
1answer
259 views

Distances for binary and non binary categorical data

I am computing a matrix of distances for categorical data. I am using the Jaccard distance since as far as I understood it should be working properly with this kind of data. I have BOTH binary and ...
1
vote
1answer
41 views

Using Mantel to explore relationship between geographic distance and a multivariate character

I'm working with bird songs. A song is composed of many vocal parameters [highest frequency (Hz), lower frequency(Hz), bandwidth(Hz), duration (s), number of notes, and son on....] I'm interested in ...
0
votes
1answer
61 views

Mantel Test data assumptions

Does the Mantel Test works with non-normal distributed samples? I couldn't find anything clear enough about it.
1
vote
0answers
22 views

How do you deal with different distance based features?

If I have a model where the set of features where a cosign distance measure makes sense for some of the features, and a Euclidean distance measure makes sense for the others for example using a BOW ...
0
votes
1answer
87 views

Order of Matrix Operations in Mahalanobis Calculations

I'm teaching myself to translate equations to code after many years of letting my math skills atrophy, and am trying to do it on my own as much as possible. I've run into a couple of difficult ...
0
votes
0answers
122 views

Using relative frequency for Euclidean and cosine distance (dissimilarity)

How to calculate the Euclidean distance (dissimilarity) between two documents, e.g., D1 and D2 using relative frequency? Here is an example of both cosine and Euclidean distance between two ...
0
votes
0answers
98 views

Multidimensional Scaling: Interpreting output of different distance matrices (Euclidean or correlation)

I would like to understand the difference between using a Euclidean distance matrix or correlation matrix as input to a nMDS algorithm. I have completed MDS plots of both, and while similar, the ...
1
vote
0answers
16 views

Distribution of altered Mahalanobis distance [duplicate]

Let's say I have a set of i.i.d. samples $X_1,\ldots,X_N \sim N_p(\mu, \Sigma)$. Now define \begin{equation} d^2_i(b)=(X_i - b)'\Sigma^{-1}(X_i-b) \end{equation} which is essentially the Mahalanobis ...