Distance functions refer to functions used for quantifying the notion of distance between members of a set, or between objects.

learn more… | top users | synonyms

0
votes
0answers
10 views

Gaussian Mixture Model with Custom Distance Metric

I have some 1D data that I want to cluster using Mixture of Gaussian. However, the data "wraps around" at two extremes. Specifically, I have a list of angles from $-\pi$ to $\pi$ and the data near two ...
0
votes
0answers
21 views

Can I compare Mahalanobis distances from different distributions?

I have a multivariate dataset representing multiple locations, each of which has a set of reference observations and a single test observation. For each location, I would like to measure how anomalous ...
3
votes
2answers
49 views

How to measure similarity of bivariate probability distributions?

I have three different distributions of 2D data: or Now I like to know whether distribution two is more similar to distribution one (2 to 1) than distribution three is to distribution one (3 to ...
0
votes
0answers
16 views

evaluate the similarity between two time series

I have two time series, $\mathcal{T}_1$ and $\mathcal{T}_2$, each time series is of two dimensional. One time series is collected from two sensors (SA, SB), and the other is collected from other ...
1
vote
0answers
48 views

Hierarchical clustering, linkage methods and dynamic time warping

My goal is to cluster time series based on their DTW distance. Therefore I've calculated full distance matrices as input for several clustering algorithms. I first had a look at hierarchical methods, ...
1
vote
0answers
19 views

handling categorical data with a large amount of categories

I have data containing few categorical columns with a huge amount of categories at each (more than 1000 different categories at each column). I have to build a predictive model on this data, using the ...
1
vote
3answers
79 views

Ask for suggestions on clustering methods on a large dataset with mixed types of variables

I need to build segmentation on a large customer dataset with more than 300K records and many variables, including continuous like income and age, ordinal like education level and membership level, ...
2
votes
0answers
12 views

Approximation of objective based on statistical distance

I am a computer science researcher (mostly theoretical) currently in midst of statistics and not able to figure out how to proceed. At an abstract level, I have a hypothesis for an unknown ...
1
vote
0answers
54 views

Comparing two vectors with the same numbers in different positions [closed]

I have two vectors of the same size representing one score for different sites (x and y coordinates) under distinct situations (each vector is one situation). At each vector the scores only change ...
0
votes
0answers
38 views

comparing probability histogram

I have two probability histogram samples. I know there are methods(i.e KS test etc) out there to compare histograms but I am trying to compare through simple sum of absolute difference between these ...
1
vote
1answer
148 views

If Manhattan distance always performs better on a dataset…what does it mean?

I'm analyzing my dataset using kNN. I experimented with various distance functions but Manhattan seems to perform better in terms of lowest RMSE over various values of k. I've read a bit about ...
1
vote
0answers
35 views

Distance metrics in “high” dimensions

Thanks in advance: this is a very nice site. This question has been made more specific in response to a comment. original question ----------------- Reading the various sources concerning distance ...
0
votes
2answers
94 views

Mahalanobis distance measure for clustering

Let's say I have a group of clusters. Would you recommend Mahalanobis distance measure for checking if new arrived data belongs to existing clusters or it is an outlier? Also, would you recommend ...
0
votes
1answer
43 views

Similarity scoring to compare multi-dimensional datasets

I am trying to come up with a mechanism of scoring a set of multidimensional datasets based on a similarity with an ideal dataset. Each dataset will all have the same dimensions along with the ideal. ...
3
votes
1answer
37 views

Can I use k-means with a distance matrix composed of percentages? [duplicate]

I have objects o1, o2,...,on and for each pair I calculate a value that measures the pair's difference. This is a percentage, so for example o1o2 differ by 56%. Now I want to cluster this data. I can ...
1
vote
0answers
79 views

Distance measure for categorical attributes for k-Nearest Neighbor

For my class project, I am working on the Kaggle competition - Don't get kicked The project is to classify test data as good/bad buy for cars. There are 34 features and the data is highly skewed. I ...
0
votes
0answers
9 views

Find input image (ID,passport) in imagesDB based on similarity

I would like to decide if image is exists on DB images (pictures of IDs,passport,Stu. card,etc) I thought of KNN alghorithem that will plot the K closest images. Options for distance metric: 1) sum ...
0
votes
1answer
18 views

Why do two identical feature vectors (distance score 0) get different labels in DBSCAN?

I have two identical feature vectors. They have a distance score of 0. I perform DBSCAN Clustering (using sci-kit) and they get different labels. Is this expected behaviour?
0
votes
1answer
33 views

Multivariate nonparametric divergence (or distance) between distributions

For example, we could say I have two fruit classes (oranges and apples) and for each one I measured different statistics of interest, for example: width, height, sugar, water... of a lot of fruit ...
1
vote
1answer
66 views

Why is k-medians typically used with Manhattan rather than Euclidean distance?

K-medians is typically used with Manhattan distance rather than Euclidean distance. Why is this?
1
vote
1answer
52 views

Calculate PCoA scores for dataframe “x”, based on the distance matrix of dataframe “y”

I'm trying to use multivariate techniques to compare two datasets (same structure) that were collected using different sampling techniques. I'd like to compute a PCoA for the first dataset (D1), and ...
0
votes
1answer
14 views

Calculating distance comparing sets of frequencies

I have two sets of items, say A (with items a1, a2..) and B (with items b1,b2..). Each item in A appears with different frequency with items in B, so each item would have a list of B items with ...
0
votes
0answers
16 views

How to split a class which is not very cohesive?

Using the silhouette width metric I can find out as to how well each object lies within its class after classification is done. I next find the average silhouette width of objects within a class and ...
2
votes
1answer
114 views

Distances for binary and non binary categorical data

I am computing a matrix of distances for categorical data. I am using the Jaccard distance since as far as I understood it should be working properly with this kind of data. I have BOTH binary and ...
1
vote
1answer
36 views

Using Mantel to explore relationship between geographic distance and a multivariate character

I'm working with bird songs. A song is composed of many vocal parameters [highest frequency (Hz), lower frequency(Hz), bandwidth(Hz), duration (s), number of notes, and son on....] I'm interested in ...
0
votes
1answer
41 views

Mantel Test data assumptions

Does the Mantel Test works with non-normal distributed samples? I couldn't find anything clear enough about it.
1
vote
0answers
20 views

How do you deal with different distance based features?

If I have a model where the set of features where a cosign distance measure makes sense for some of the features, and a Euclidean distance measure makes sense for the others for example using a BOW ...
0
votes
1answer
58 views

Order of Matrix Operations in Mahalanobis Calculations

I'm teaching myself to translate equations to code after many years of letting my math skills atrophy, and am trying to do it on my own as much as possible. I've run into a couple of difficult ...
0
votes
0answers
92 views

Using relative frequency for Euclidean and cosine distance (dissimilarity)

How to calculate the Euclidean distance (dissimilarity) between two documents, e.g., D1 and D2 using relative frequency? Here is an example of both cosine and Euclidean distance between two ...
0
votes
0answers
73 views

Multidimensional Scaling: Interpreting output of different distance matrices (Euclidean or correlation)

I would like to understand the difference between using a Euclidean distance matrix or correlation matrix as input to a nMDS algorithm. I have completed MDS plots of both, and while similar, the ...
1
vote
0answers
14 views

Distribution of altered Mahalanobis distance [duplicate]

Let's say I have a set of i.i.d. samples $X_1,\ldots,X_N \sim N_p(\mu, \Sigma)$. Now define \begin{equation} d^2_i(b)=(X_i - b)'\Sigma^{-1}(X_i-b) \end{equation} which is essentially the Mahalanobis ...
1
vote
1answer
99 views

How to choose the right distance matrix for clustering?

I am attempting simple Ward type clustering. However, the R package is proving several choices to use for the distance matrix. I am wondering how I am supposed to determine the right distance matrix ...
1
vote
1answer
87 views

Distance between two random vectors

I have two random vectors, $A$ and $B$ with each consisting of $n$ geographical co-ordinates $(x_1,y_1),(x_2,y_2)\dots (x_n,y_n)$ and $(\tilde{x}_1,\tilde{y}_1),(\tilde{x}_2,\tilde{y}_2)\dots ...
0
votes
1answer
279 views

hclust, R and Euclidean distances: weird stuff

I have a table of similarities expressed through cosines and am trying to do some cluster analysis in R, using hclust and ...
1
vote
1answer
260 views

Cook's Distance

The formula of Cook's distance is $$D_i=\frac{(\hat Y-\hat Y(i))^{\prime}(\hat Y-\hat Y(i))}{p\times MSE}$$ where, $\hat Y$ is the prediction from the full regression model and $\hat Y$ is a ...
0
votes
1answer
159 views

advantage of euclidean distance for classification

Has euclidean distance any advantage in compare to another distance based methods like Manhatan distance or Maximum difference metric?
2
votes
1answer
171 views

Jeffries Matusita distance for 14 variables

I wish to perform Jeffries-Matusita distance on 14 spectral bands. Is there anyone who can help with how it is done in R? Thank you.
4
votes
1answer
245 views

How to distance and to MDS-plot objects according their complex shape

Suppose I have four basal forms of signal (blue, purple, red, green). I also have created transition forms between each other. If you carefully look on the picture below, you can see that for example ...
1
vote
0answers
28 views

Representing a distance matrix in the plane [duplicate]

I've worked with observations as vectors with both continuous and categorical variables. In both cases one can use dimensionality reduction techniques such as PCA (in the latter case through ...
0
votes
0answers
28 views

Log likelihood - understand depper

I want to use log likelihood formula to relate between two items. The formula is: LLR = 2 sum(k) (H(k) - H(rowSums(k)) - H(colSums(k))) When this is the table: ...
2
votes
0answers
35 views

Way to Measure Groupings Using Distances Between Individuals?

I am working on a problem that requires me to measure groupings of people. I have the location of every individual in my sample at every point in time. It's therefore trivial to calculate the ...
2
votes
0answers
70 views

What is a good similarity measure to use when missing data is a significant issue?

I have a list of cities that I want to compare in terms of their similarity. Each city can described by a large but finite number of characteristics but most of them will have missing data for some ...
2
votes
0answers
42 views

Calculate number of standard deviations separating two multivariate Gaussians?

Given a set of multivariate Gaussian distributions (from fitting a Gaussian mixture model) I would like to be able to calculate the likelihood that a data point drawn from one Gaussian will improperly ...
0
votes
0answers
47 views

Histogram distance metric for extreme values only

I am interested in a histogram comparison method or histogram matching technique that takes into account only the tails of the distribution. Consider the following histograms: Histogram 1: ...
1
vote
1answer
37 views

calculate Levenshtein Distance for web click stream data

I want to calculate Levenshtein Distance between to web click paths. I have a web page list around 500. ...
1
vote
0answers
68 views

How does Gower distance work with free text?

The Gower distance measure is a good measure for mixed-type data (i.e., data attributes can be qualitative, categorical, ordinal or binary). But can data attributes be free-text (e.g., names of ...
0
votes
1answer
56 views

What is the $p$ in Cook's distance?

In the equation for Cook's distance: $$D_i = \frac{\sum_{j=1}^{n}(\hat{y}_j - \hat{y}_{j(i)})^2}{p MSE}$$ the value of $p$ is defined as "the number of fitted parameters in the model." What does ...
68
votes
6answers
7k views

Why is Euclidean distance not a good metric in high dimensions?

I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
0
votes
2answers
61 views

Is this a valid use case for Euclidean distance?

I have a set of points which is a count of links that users have clicked on : ...
0
votes
0answers
166 views

Negative Mahalanobis Distance

I would like to calculate a compound scores of several normal distributed continues standardized (z-score) variables. Some of these measures are correlated, some are not. Hence, I would like to take ...