Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

learn more… | top users | synonyms

0
votes
0answers
13 views

Is it possible to scale categorical variable before calculating a distance matrix?

My ultimate goal is to find similar customers by comparing characteristics of non-customers to existing customers. The characteristics are mostly non-numeric. My hope was to scale(data) and then ...
1
vote
0answers
8 views

Compute the infimum of the L^2-distance to the set of Gaussian functions [migrated]

I am faced with the following question: given a (nonnegative) distribution function $f$ over $\mathbb{R}$ with unit mass and $\int_{\mathbb{R}}f(x)x^2dx=\sigma^2$ given, I am trying to find the ...
3
votes
0answers
34 views

MDS: Is Kruskal's Stress-1 affected by scale of the data, or the number of points?

In Multidimensional Scaling, Kruskal's Stress-1 is a commonly used measure of fit. It is defined as: $\sqrt{\frac{\sum (d_{ij}-\delta_{ij})^{2}}{\sum d_{ij}^{2}}}$ where $d_{ij}$ represents the ...
3
votes
1answer
27 views

Difference between languages (spoken)?

I'm trying to perform a hierarchical clustering, to aggregate some "zones" or neighborhoods of a city, based on the language that is used most in that zone In order to do so, I have at hand a dataset ...
0
votes
0answers
11 views

What family of GLM when response is Bray-Curtis dissimilarity? Should I use “adonis” instead?

I have some questions about testing for effects of different experimental levels on community similarity. I'll explain my planned experimental design before I ask the questions: I have two methods ...
0
votes
0answers
8 views

Combine several measures to one using mahalanobis or euclidean distance

I have 7 financial variables (ratios and simple interval variables), which I want to combine in one measure. I will further use it the regression analysis.The aggregate measure has to account for the ...
0
votes
0answers
8 views

Hierarchical classification distance based measures

I have sentences that have been assigned a class. The class belongs to a hierarchy. A sentence has one class each. I've performed flat classification experiments and returned some predicted classes on ...
2
votes
1answer
26 views

Is there a multi-Gaussian version of the Mahalanobis distance ?

Let's say we are in an $N$-dimensional space and that we have a large set of data. The distribution of this $N$-dimensional point cloud can be modeled by a multivariate Gaussian mixture model ...
3
votes
0answers
19 views

A distance measure for intervals in [0,1] [closed]

We are carrying out an experiment about automatic generation of fuzzy synsets and, as a result of the experiment, we have a lot of closed intervals in [0,1] which we need to compare (between them and ...
0
votes
0answers
6 views

How to determine a formula for an index of adherence

I hope you will be patient with the inarticulate question of a non-mathematician. It's hard to get an answer when you don't even know how to ask the question. Here the contest: Let's say that I have ...
0
votes
0answers
10 views

python: how to use cholesky decomposition for whiting the feature matrix before k-means

I would like to use scipy.linalg.cholesky before scipy.cluster.vq.kmeans2 so the clustering will be on the "Mahalanobis" ...
2
votes
1answer
40 views

Evaluating two sets of random samples

Let $p$ be a probability distribution that can be computed tractably for any given point. I use two MCMC methods to generate samples from the distributions. For each MCMC method, I run 1000 Markov ...
0
votes
0answers
28 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
0
votes
0answers
39 views

Calculate real-world disparity from disparity map

I am trying to estimate the distance moved by a car using stereo images captured from cameras mounted on a car. For this, I have planned on getting the depth to an object at time t0 and then get the ...
1
vote
0answers
53 views

Distance between random variables [closed]

I have found plenty of ways to compute the distance between random variables. However, I did not find any taking something else than the random variables as input. Do you know whether or not there is ...
0
votes
0answers
18 views

Compute probability from distance-score

I compute Euclidian distances between a point I want to analyze and a set of points I have. I want to sort my points by descreasing "similarity". I used to compute a "score" by inverting the distance ...
0
votes
1answer
52 views

distance measure of two discrete probability histograms (distance between two vectors)

I have multiple sets of discrete probability histograms(vectors) and I want to measure the distance between each histogram. I have done some research but I am in doubt. Literature suggest I could ...
1
vote
0answers
23 views

Selecting correct settings for the order of Minkowski distance

I am looking to compute the distance between vectors of word frequencies (and I am new to this). I am trying out the Minkowski distance as implemented in Scipy. The documentation asks me to specify a ...
1
vote
0answers
12 views

Does maximal Bhattacharyya coefficient imply mimimal total variation distance?

Some context: I`m working on numerical optimization (linear programming), on probability distributions denoted P,Q. We want to find the minimal total variation distance and maximal Bhattacharyya ...
1
vote
1answer
37 views

Get distance matrix directly condensed

I am developing a content-recommender Python system and most of my items (~8 millions) are static so I have thought about pre-computing the top 150 similar items for each item. This way, when a user ...
5
votes
1answer
57 views

Finding optimal correspondences between objects given two square distance matrices

I would like to find the optimal correspondences between two systems of objects based on the distances between objects WITHIN the two systems. So, the input to the algorithm would be two square ...
1
vote
0answers
19 views

Hamming distance of Bernoulli RV

Assume you draw $k$ data points out of $n$ data points. Each data point is composed of $m$ Bernoulli random variables. You may assume that the data points are i.i.d and likewise their coordinates, but ...
3
votes
1answer
35 views

Distribution-free test for two-sample multivariate distributions

Suppose that $X_1, \cdots, X_n$ and $Y_1, \cdots, Y_n$ are samples of $R^d$ vectors with distributions $X$ and $Y$ respectively. In addition, assume that there is one-to-one mapping between the first ...
2
votes
1answer
14 views

How to deal with non-independent data when comparing populations across a range of distances

I am trying to figure out how to carry out an analysis but am having trouble finding any information. I am interested in finding out whether values of sensitivity across populations are more similar ...
0
votes
0answers
35 views

Kolmogorov-Smirnov test between two distributions - R

I've two distributions computed on the same grid (that is, for each point of the grid I know the value of each CDF at that point). I want to check whether the two distributions are the same. I can't ...
1
vote
0answers
22 views

Distance Estimation from Signal Level?

So, I want to learn about machine learning and apply it to my project. I have set of data which includes position of a car and unknown emitter signal level. I have to estimate the distance based on ...
0
votes
0answers
20 views

Metric from a positive definite matrix

I'm trying to prove that the Mahalanobis distance is an actual distance, more in general Given B symmetric and positive definite matrix set d(x,y)=(x-y)'B(x-y) ( ...
0
votes
0answers
26 views

Elbow method implementation for hierarchical clustering

I've got a dataset that I need to divide intro clusters using hierarchical clustering algorithm. I've decided to try to employ an Elbow Method as a way of determining optimal no. of clusters k. ...
0
votes
0answers
15 views

Kullback-Leibler and Battacharyya divergences between Hidden Markov Models with discrete emissions

Im trying to figure out how to compute KL or Battacharyya divergences between two HMMs models. I found papers which are about HMMs with normaly distributed emissions, but nothing for discrete ...
0
votes
0answers
19 views

Getting the closest observations to a category

I've been thinking for too long on this already and I decided to socialize my problem to see what I can do. My set up is the following: the observations are already classified by a questionary (V1 to ...
0
votes
0answers
25 views

Clustering subjects regarding binary property vectors

My data set consists of 120 subjects and 50 binary attributes. I want to find clusters in that. So far I started by a visual analysis plotting the n x n matrix taking the several similarity measures ...
0
votes
1answer
54 views

How to soften or mitigate vector similarity measure?

I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located. I construct two sets of nine concentric and equidistant circles with centers ...
0
votes
0answers
10 views

Hypothesis Testing With Distance Measurements - Variance Calculation

I am looking at an existing method that is making decisions based on hypothesis testing using a distance measurement. The test looks like a Z- or T-test, where the computed distance and its variance ...
1
vote
1answer
71 views

Proving that cosine distance function defined by cosine similarity between two unit vectors does not satisfy triangle inequality

How to prove that the cosine distance function defined by cosine similarity between two unit vectors does not satisfy the triangle inequality?
1
vote
1answer
27 views

Calculating “distances” from quantiles

I am trying to compare the measurements from two different methods, A and B, that have each reported 16%, 50%, and 84% quantiles for a measurement X. How can I capture in a number how far away A and B ...
0
votes
1answer
24 views

Distance for fuzzy c-mean clustering

Fuzzy c-means clustering will use Euclidean distance and the mean square error, or Manhattan distance and the mean absolute error. Which of those distance measures you should use for fuzzy c-means, ...
1
vote
1answer
22 views

Calculate probability of distance for d-dimensional normal

Is there any simple way to calculate the probability of distance in the following form for d-dimensional normal distribution? $P(||\mathbf{x}-\mathbf{\mu}||^2>||\mathbf{x}-\mathbf{a}||^2)$, where ...
0
votes
1answer
60 views

How to apply distance metrics to compare bar plot (nominal histogram) data

I have a data set for libraries, I would like to find the (Similarity / dissimilarity) among it based on book category, so for each category there is single value represent the number of books that ...
0
votes
1answer
62 views

Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?

I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
1
vote
0answers
21 views

Ranking a negative correlation as equal to a positive correlation on a column-by-column basis for distance measures

I am very new to r, but have managed to muddle together a functional script to tackle data from a screen I am working on. I have a list of mutants of genes in a signaling network and values (OD550) ...
0
votes
0answers
35 views

Choosing the right linkage method for hierarchical clustering

I am performing hierarchical clustering on data I've gathered and processed from the reddit data dump on Google BigQuery. My process is the following: Get the latest 1000 posts in /r/politics ...
0
votes
0answers
16 views

Separable Kernel Density Estimate

Thank you for reading my question. Before I begin, I am no mathematician and so any help/pointers are very welcome. I have been reading the following paper: ...
0
votes
1answer
124 views

A question on cosine similarity & k-means

I used the following code to perform clustering of a dataset in R. distMatrix1 <- dist(sample2, method="cosine") km<-kmeans(distMatrix1,3) I have got some ...
-1
votes
2answers
112 views

How to calculate distance between points in DBSCAN matrix data? [closed]

I'm making a simple C implementation of DBSCAN following his pseudocode. If I well underand how DBSCAN works, I may represent my set of N elements (each with M features) with a NxM matrix. When it ...
0
votes
0answers
36 views

Identifying the most representative observation in a sample

Given a multidimensional X, is there a standard way of identifying the most representative observation? A few options come to mind: Minimum average distance (e.g. cosine, Mahalanobis) from all other ...
0
votes
0answers
24 views

Measuring distances between numbers, in a way

I'm using fantasy football data from Draft Kings. I have data frames that look like this in r: ...
1
vote
0answers
25 views

Average distance in distance matrix

I have a set of many longitude and latitude points within a city. I constructed the nxn euclidean distance matrix. My goal is to know which is the average distance ...
1
vote
1answer
36 views

Bhattacharya Distance on Distributions (Matrices) with Different Number of Variables (Dimensions) [duplicate]

We have two matrices, $A$ and $B$, representing two different probability distributions, with dimensions, $m*n$ and $k*n$, respectively. How can we calculate the Bhattacharya distance or another ...
1
vote
1answer
24 views

Can I use Bray-Curtis distance when performing MRPP or MRBP?

Can I use Bray-Curtis distance when performing MRPP (Multi-response Permutation Procedures) or MRBP (Blocked Multi-response Permutation Procedures)? [See Ch 24 of McCune & Grace (2002) for ...
1
vote
0answers
33 views

Intuitive explanation for covariance and inverse covariance

Say we are given n observations from the same multidimensional distribution $D$. I am trying to understand what is the intuition behind the following norms: $\sqrt{x^TC_nx}$ and $\sqrt{x^TC_n^{-1}x}$ ...