Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

learn more… | top users | synonyms

0
votes
0answers
13 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
0
votes
0answers
25 views

hierarchical clustering on rows of varying length with sequence of numbers [on hold]

I want to do hierarchical clustering in one of my project. My original problem is that I have a huge graph on which I have iterated large number of paths and reported nodes of path in below format. ...
0
votes
0answers
17 views

Clustering words by similarity and occurance

I have the following problem: I have a list of document terms and their frequency. These terms are not common English words. Numerous variants (i.e. spelling mistakes) relating to these terms are ...
0
votes
0answers
29 views

Calculate real-world disparity from disparity map

I am trying to estimate the distance moved by a car using stereo images captured from cameras mounted on a car. For this, I have planned on getting the depth to an object at time t0 and then get the ...
1
vote
0answers
50 views

Distance between random variables [closed]

I have found plenty of ways to compute the distance between random variables. However, I did not find any taking something else than the random variables as input. Do you know whether or not there is ...
0
votes
0answers
18 views

Compute probability from distance-score

I compute Euclidian distances between a point I want to analyze and a set of points I have. I want to sort my points by descreasing "similarity". I used to compute a "score" by inverting the distance ...
0
votes
1answer
38 views

distance measure of two discrete probability histograms (distance between two vectors)

I have multiple sets of discrete probability histograms(vectors) and I want to measure the distance between each histogram. I have done some research but I am in doubt. Literature suggest I could ...
1
vote
0answers
17 views

Selecting correct settings for the order of Minkowski distance

I am looking to compute the distance between vectors of word frequencies (and I am new to this). I am trying out the Minkowski distance as implemented in Scipy. The documentation asks me to specify a ...
1
vote
0answers
10 views

Does maximal Bhattacharyya coefficient imply mimimal total variation distance?

Some context: I`m working on numerical optimization (linear programming), on probability distributions denoted P,Q. We want to find the minimal total variation distance and maximal Bhattacharyya ...
1
vote
1answer
32 views

Get distance matrix directly condensed

I am developing a content-recommender Python system and most of my items (~8 millions) are static so I have thought about pre-computing the top 150 similar items for each item. This way, when a user ...
-1
votes
0answers
10 views

Levenshtein string distance with only appending and prepending but no insertion

Is there a proper name for this distance? For example ...
-1
votes
0answers
12 views

distance metric that preserves direction

I was wondering if there is a distance metric that preserves direction? I know the euclidean metric is simply an absolute distance, but I want to see if perhaps some points are in a certain ...
5
votes
1answer
54 views

Finding optimal correspondences between objects given two square distance matrices

I would like to find the optimal correspondences between two systems of objects based on the distances between objects WITHIN the two systems. So, the input to the algorithm would be two square ...
1
vote
0answers
19 views

Hamming distance of Bernoulli RV

Assume you draw $k$ data points out of $n$ data points. Each data point is composed of $m$ Bernoulli random variables. You may assume that the data points are i.i.d and likewise their coordinates, but ...
3
votes
1answer
33 views

Distribution-free test for two-sample multivariate distributions

Suppose that $X_1, \cdots, X_n$ and $Y_1, \cdots, Y_n$ are samples of $R^d$ vectors with distributions $X$ and $Y$ respectively. In addition, assume that there is one-to-one mapping between the first ...
2
votes
1answer
12 views

How to deal with non-independent data when comparing populations across a range of distances

I am trying to figure out how to carry out an analysis but am having trouble finding any information. I am interested in finding out whether values of sensitivity across populations are more similar ...
0
votes
0answers
30 views

Kolmogorov-Smirnov test between two distributions - R

I've two distributions computed on the same grid (that is, for each point of the grid I know the value of each CDF at that point). I want to check whether the two distributions are the same. I can't ...
1
vote
0answers
22 views

Distance Estimation from Signal Level?

So, I want to learn about machine learning and apply it to my project. I have set of data which includes position of a car and unknown emitter signal level. I have to estimate the distance based on ...
0
votes
0answers
18 views

Metric from a positive definite matrix

I'm trying to prove that the Mahalanobis distance is an actual distance, more in general Given B symmetric and positive definite matrix set d(x,y)=(x-y)'B(x-y) ( ...
0
votes
0answers
21 views

Elbow method implementation for hierarchical clustering

I've got a dataset that I need to divide intro clusters using hierarchical clustering algorithm. I've decided to try to employ an Elbow Method as a way of determining optimal no. of clusters k. ...
0
votes
0answers
14 views

Kullback-Leibler and Battacharyya divergences between Hidden Markov Models with discrete emissions

Im trying to figure out how to compute KL or Battacharyya divergences between two HMMs models. I found papers which are about HMMs with normaly distributed emissions, but nothing for discrete ...
0
votes
0answers
19 views

Getting the closest observations to a category

I've been thinking for too long on this already and I decided to socialize my problem to see what I can do. My set up is the following: the observations are already classified by a questionary (V1 to ...
0
votes
0answers
25 views

Clustering subjects regarding binary property vectors

My data set consists of 120 subjects and 50 binary attributes. I want to find clusters in that. So far I started by a visual analysis plotting the n x n matrix taking the several similarity measures ...
0
votes
1answer
53 views

How to soften or mitigate vector similarity measure?

I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located. I construct two sets of nine concentric and equidistant circles with centers ...
0
votes
0answers
9 views

Hypothesis Testing With Distance Measurements - Variance Calculation

I am looking at an existing method that is making decisions based on hypothesis testing using a distance measurement. The test looks like a Z- or T-test, where the computed distance and its variance ...
1
vote
1answer
63 views

Proving that cosine distance function defined by cosine similarity between two unit vectors does not satisfy triangle inequality

How to prove that the cosine distance function defined by cosine similarity between two unit vectors does not satisfy the triangle inequality?
1
vote
1answer
23 views

Calculating “distances” from quantiles

I am trying to compare the measurements from two different methods, A and B, that have each reported 16%, 50%, and 84% quantiles for a measurement X. How can I capture in a number how far away A and B ...
0
votes
1answer
16 views

Distance for fuzzy c-mean clustering

Fuzzy c-means clustering will use Euclidean distance and the mean square error, or Manhattan distance and the mean absolute error. Which of those distance measures you should use for fuzzy c-means, ...
1
vote
1answer
22 views

Calculate probability of distance for d-dimensional normal

Is there any simple way to calculate the probability of distance in the following form for d-dimensional normal distribution? $P(||\mathbf{x}-\mathbf{\mu}||^2>||\mathbf{x}-\mathbf{a}||^2)$, where ...
0
votes
1answer
58 views

How to apply distance metrics to compare bar plot (nominal histogram) data

I have a data set for libraries, I would like to find the (Similarity / dissimilarity) among it based on book category, so for each category there is single value represent the number of books that ...
0
votes
1answer
50 views

Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?

I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
1
vote
0answers
19 views

Ranking a negative correlation as equal to a positive correlation on a column-by-column basis for distance measures

I am very new to r, but have managed to muddle together a functional script to tackle data from a screen I am working on. I have a list of mutants of genes in a signaling network and values (OD550) ...
0
votes
0answers
32 views

Choosing the right linkage method for hierarchical clustering

I am performing hierarchical clustering on data I've gathered and processed from the reddit data dump on Google BigQuery. My process is the following: Get the latest 1000 posts in /r/politics ...
0
votes
0answers
16 views

Separable Kernel Density Estimate

Thank you for reading my question. Before I begin, I am no mathematician and so any help/pointers are very welcome. I have been reading the following paper: ...
0
votes
1answer
105 views

A question on cosine similarity & k-means

I used the following code to perform clustering of a dataset in R. distMatrix1 <- dist(sample2, method="cosine") km<-kmeans(distMatrix1,3) I have got some ...
-1
votes
2answers
99 views

How to calculate distance between points in DBSCAN matrix data? [closed]

I'm making a simple C implementation of DBSCAN following his pseudocode. If I well underand how DBSCAN works, I may represent my set of N elements (each with M features) with a NxM matrix. When it ...
0
votes
0answers
34 views

Identifying the most representative observation in a sample

Given a multidimensional X, is there a standard way of identifying the most representative observation? A few options come to mind: Minimum average distance (e.g. cosine, Mahalanobis) from all other ...
0
votes
0answers
24 views

Measuring distances between numbers, in a way

I'm using fantasy football data from Draft Kings. I have data frames that look like this in r: ...
1
vote
0answers
24 views

Average distance in distance matrix

I have a set of many longitude and latitude points within a city. I constructed the nxn euclidean distance matrix. My goal is to know which is the average distance ...
1
vote
1answer
34 views

Bhattacharya Distance on Distributions (Matrices) with Different Number of Variables (Dimensions) [duplicate]

We have two matrices, $A$ and $B$, representing two different probability distributions, with dimensions, $m*n$ and $k*n$, respectively. How can we calculate the Bhattacharya distance or another ...
1
vote
1answer
18 views

Can I use Bray-Curtis distance when performing MRPP or MRBP?

Can I use Bray-Curtis distance when performing MRPP (Multi-response Permutation Procedures) or MRBP (Blocked Multi-response Permutation Procedures)? [See Ch 24 of McCune & Grace (2002) for ...
1
vote
0answers
32 views

Intuitive explanation for covariance and inverse covariance

Say we are given n observations from the same multidimensional distribution $D$. I am trying to understand what is the intuition behind the following norms: $\sqrt{x^TC_nx}$ and $\sqrt{x^TC_n^{-1}x}$ ...
1
vote
0answers
43 views

Log-likelihood distance

How to calculate log-likelihood distance between clusters in two step clustering? if the following is the solution,then how to proceed? I would appreciate if someone can help me to find this. I am ...
0
votes
1answer
51 views

What is the use of distance matrix in clustering algorithms?

I found a C library for clustering and I was reading about the distance matrix here: it says: The first step in clustering problems is usually to calculate the distance matrix. This matrix ...
3
votes
1answer
132 views

Visualizing Mahalanobis distance in more than 3 dimensions

I have 4 dimensional data in a matrix, group1, that looks like the following: ...
0
votes
0answers
7 views

Can I use Bray-Curtis similarity with environmental data? [duplicate]

I have 18 metals concentration (in ppm) in 300 soil samples. Can I use Bray-Curtis in stead of Euclidean-distance and do I need to normalize the data?
0
votes
0answers
56 views

mahalanobis Distance between 2 groups in r

I have two groups, that each group has 3 variables such as following: ...
0
votes
1answer
138 views

Why is it bad to use Pearson distance in K-means clustering? [duplicate]

I have implemented this algorithm in MATLAB and when I produce plots I notice that using Euclidean distance, I usually get presented with a clear pattern (sum of squares decreases with the number of ...
0
votes
0answers
12 views

Investigate clusters of outcome variables in a study and verify whether stratification for an exposure changes the clusters

We have a study in which we have a number of outcomes (~30) and big number of observations/patients (> 1000) and we tested the effect of a certain exposure on the probability of these outcomes. For ...
2
votes
0answers
82 views

Combining Bhattacharya Distance (or A Measure of Similarity) — across Different Variables (Properties)

We have a series of observations of different properties (such as heart rate or blood sugar level and others as well) across different days from different people from different geographical regions. ...