Measure of distance between distributions or variables, such as Euclidean distance between points in n-space.

learn more… | top users | synonyms

1
vote
1answer
15 views

Is it valid to reduce the AICc penalty for multiple variables, when some variables should be grouped?

I have a data set of mean trait values for each of 18 populations, and want to test whether several ecological variables are related to variation in traits. I'm using the corrected Akaike information ...
0
votes
0answers
13 views

Mahalanobis Function with test and train data

I am new to R programming. I was given a task at work with minimal explaination. I have a huge Data set. Can someone explain the use of Test data and train data? How to use that along with a ...
1
vote
0answers
19 views

Permanova for microbial ecology using adonis (R): should I include block?

I know this is a very basic question, but I have been quite unsuccessful in finding a good solution to the following issue. First the data. My spreadsheet is too big, so here is a fictional example: ...
0
votes
0answers
12 views

Cosine distance between two vectors containing positive as well as negative elements

I have two vectors whose elements $\in \mathbb{R}$. Everywhere I have seen the usual definition of cosine similarity and the distance = 1 - similarity. However, on ...
1
vote
0answers
17 views

estimate NND in the cluster with missing data [closed]

Would very appreciate any comments on the following problem: Imagine one has N sets of elements (points on the plane), coordinates (x,y) of 30% of the points in each set are known. So if we have 30 ...
1
vote
0answers
53 views

Euclidean distance between the unbiased covariance estimator and the true covariance

Let X be a n*p matrix whose p columns are sampled from a centred normal multivariate distribution with true covariance matrix $\Sigma$ and let $\Sigma_{n}$ be its unbiased sample covariance estimator ...
0
votes
1answer
49 views

Does it make sense to cluster based on Euclidean distances between rows of a cosine matrix?

I calculated the Cosine distances for binary data and got the relations between different variables. I need to cluster them. I tried passing the cosine matrix directly to the (clustering) function ...
1
vote
0answers
26 views

How to calculate the distance in KNN for mixed data types?

when the data is from different types (numerical and categorical) of course euclidean distance alone or hamming distance alone can't help. so i have 2 approaches: standardize all the data with ...
1
vote
1answer
29 views

How Gower's dissimilarity handle missing values in numeric columns?

I would like to ask a question about Gower dissimilarity, I was wondering how Gower measure handle missing values in numeric columns, especially that Gower standardized each column based on the range ...
0
votes
0answers
19 views

Nearest neighbor symmetric matrix

I have a k nearest neighbors (kNN) distance matrix for $n$ samples $S_n$, which is a sparse, almost-symmetric matrix with the property $x_{ij} ∈ \{0, x_{ji}\}~∀~i, j ∈ \{1,...,n\}$. A matrix entry $x_{...
0
votes
0answers
11 views

How can the shortest distance between 2 points be a curve? [migrated]

please help me with my question!! I have been asked this question by my teacher and NO NONE knows
0
votes
0answers
37 views
0
votes
0answers
14 views

Is it possible to scale categorical variable before calculating a distance matrix?

My ultimate goal is to find similar customers by comparing characteristics of non-customers to existing customers. The characteristics are mostly non-numeric. My hope was to scale(data) and then ...
4
votes
1answer
93 views

MDS: Is Kruskal's Stress-1 affected by scale of the data, or the number of points?

In Multidimensional Scaling, Kruskal's Stress-1 is a commonly used measure of fit. It is defined as: $\sqrt{\frac{\sum (d_{ij}-\delta_{ij})^{2}}{\sum d_{ij}^{2}}}$ where $d_{ij}$ represents the ...
3
votes
1answer
27 views

Difference between languages (spoken)?

I'm trying to perform a hierarchical clustering, to aggregate some "zones" or neighborhoods of a city, based on the language that is used most in that zone In order to do so, I have at hand a dataset ...
0
votes
0answers
17 views

What family of GLM when response is Bray-Curtis dissimilarity? Should I use “adonis” instead?

I have some questions about testing for effects of different experimental levels on community similarity. I'll explain my planned experimental design before I ask the questions: I have two methods ...
0
votes
0answers
8 views

Combine several measures to one using mahalanobis or euclidean distance

I have 7 financial variables (ratios and simple interval variables), which I want to combine in one measure. I will further use it the regression analysis.The aggregate measure has to account for the ...
0
votes
0answers
9 views

Hierarchical classification distance based measures

I have sentences that have been assigned a class. The class belongs to a hierarchy. A sentence has one class each. I've performed flat classification experiments and returned some predicted classes on ...
2
votes
1answer
28 views

Is there a multi-Gaussian version of the Mahalanobis distance ?

Let's say we are in an $N$-dimensional space and that we have a large set of data. The distribution of this $N$-dimensional point cloud can be modeled by a multivariate Gaussian mixture model (...
3
votes
0answers
20 views

A distance measure for intervals in [0,1] [closed]

We are carrying out an experiment about automatic generation of fuzzy synsets and, as a result of the experiment, we have a lot of closed intervals in [0,1] which we need to compare (between them and ...
0
votes
0answers
6 views

How to determine a formula for an index of adherence

I hope you will be patient with the inarticulate question of a non-mathematician. It's hard to get an answer when you don't even know how to ask the question. Here the contest: Let's say that I have ...
0
votes
0answers
17 views

python: how to use cholesky decomposition for whiting the feature matrix before k-means

I would like to use scipy.linalg.cholesky before scipy.cluster.vq.kmeans2 so the clustering will be on the "Mahalanobis" ...
2
votes
1answer
40 views

Evaluating two sets of random samples

Let $p$ be a probability distribution that can be computed tractably for any given point. I use two MCMC methods to generate samples from the distributions. For each MCMC method, I run 1000 Markov ...
0
votes
0answers
28 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
0
votes
0answers
51 views

Calculate real-world disparity from disparity map

I am trying to estimate the distance moved by a car using stereo images captured from cameras mounted on a car. For this, I have planned on getting the depth to an object at time t0 and then get the ...
1
vote
0answers
53 views

Distance between random variables [closed]

I have found plenty of ways to compute the distance between random variables. However, I did not find any taking something else than the random variables as input. Do you know whether or not there is ...
0
votes
0answers
22 views

Compute probability from distance-score

I compute Euclidian distances between a point I want to analyze and a set of points I have. I want to sort my points by descreasing "similarity". I used to compute a "score" by inverting the distance ...
0
votes
1answer
67 views

distance measure of two discrete probability histograms (distance between two vectors)

I have multiple sets of discrete probability histograms(vectors) and I want to measure the distance between each histogram. I have done some research but I am in doubt. Literature suggest I could ...
1
vote
0answers
28 views

Selecting correct settings for the order of Minkowski distance

I am looking to compute the distance between vectors of word frequencies (and I am new to this). I am trying out the Minkowski distance as implemented in Scipy. The documentation asks me to specify a "...
1
vote
0answers
15 views

Does maximal Bhattacharyya coefficient imply mimimal total variation distance?

Some context: I`m working on numerical optimization (linear programming), on probability distributions denoted P,Q. We want to find the minimal total variation distance and maximal Bhattacharyya ...
1
vote
1answer
43 views

Get distance matrix directly condensed

I am developing a content-recommender Python system and most of my items (~8 millions) are static so I have thought about pre-computing the top 150 similar items for each item. This way, when a user ...
5
votes
1answer
61 views

Finding optimal correspondences between objects given two square distance matrices

I would like to find the optimal correspondences between two systems of objects based on the distances between objects WITHIN the two systems. So, the input to the algorithm would be two square ...
1
vote
0answers
19 views

Hamming distance of Bernoulli RV

Assume you draw $k$ data points out of $n$ data points. Each data point is composed of $m$ Bernoulli random variables. You may assume that the data points are i.i.d and likewise their coordinates, but ...
3
votes
1answer
35 views

Distribution-free test for two-sample multivariate distributions

Suppose that $X_1, \cdots, X_n$ and $Y_1, \cdots, Y_n$ are samples of $R^d$ vectors with distributions $X$ and $Y$ respectively. In addition, assume that there is one-to-one mapping between the first ...
2
votes
1answer
14 views

How to deal with non-independent data when comparing populations across a range of distances

I am trying to figure out how to carry out an analysis but am having trouble finding any information. I am interested in finding out whether values of sensitivity across populations are more similar ...
0
votes
0answers
35 views

Kolmogorov-Smirnov test between two distributions - R

I've two distributions computed on the same grid (that is, for each point of the grid I know the value of each CDF at that point). I want to check whether the two distributions are the same. I can't ...
1
vote
0answers
27 views

Distance Estimation from Signal Level?

So, I want to learn about machine learning and apply it to my project. I have set of data which includes position of a car and unknown emitter signal level. I have to estimate the distance based on ...
0
votes
0answers
20 views

Metric from a positive definite matrix

I'm trying to prove that the Mahalanobis distance is an actual distance, more in general Given B symmetric and positive definite matrix set d(x,y)=(x-y)'B(x-y) ( ...
0
votes
0answers
28 views

Elbow method implementation for hierarchical clustering

I've got a dataset that I need to divide intro clusters using hierarchical clustering algorithm. I've decided to try to employ an Elbow Method as a way of determining optimal no. of clusters k. ...
0
votes
0answers
15 views

Kullback-Leibler and Battacharyya divergences between Hidden Markov Models with discrete emissions

Im trying to figure out how to compute KL or Battacharyya divergences between two HMMs models. I found papers which are about HMMs with normaly distributed emissions, but nothing for discrete ...
0
votes
0answers
19 views

Getting the closest observations to a category

I've been thinking for too long on this already and I decided to socialize my problem to see what I can do. My set up is the following: the observations are already classified by a questionary (V1 to ...
0
votes
0answers
28 views

Clustering subjects regarding binary property vectors

My data set consists of 120 subjects and 50 binary attributes. I want to find clusters in that. So far I started by a visual analysis plotting the n x n matrix taking the several similarity measures (...
0
votes
1answer
54 views

How to soften or mitigate vector similarity measure?

I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located. I construct two sets of nine concentric and equidistant circles with centers ...
0
votes
0answers
13 views

Hypothesis Testing With Distance Measurements - Variance Calculation

I am looking at an existing method that is making decisions based on hypothesis testing using a distance measurement. The test looks like a Z- or T-test, where the computed distance and its variance ...
1
vote
1answer
81 views

Proving that cosine distance function defined by cosine similarity between two unit vectors does not satisfy triangle inequality

How to prove that the cosine distance function defined by cosine similarity between two unit vectors does not satisfy the triangle inequality?
1
vote
1answer
28 views

Calculating “distances” from quantiles

I am trying to compare the measurements from two different methods, A and B, that have each reported 16%, 50%, and 84% quantiles for a measurement X. How can I capture in a number how far away A and B ...
0
votes
1answer
26 views

Distance for fuzzy c-mean clustering

Fuzzy c-means clustering will use Euclidean distance and the mean square error, or Manhattan distance and the mean absolute error. Which of those distance measures you should use for fuzzy c-means, ...
1
vote
1answer
22 views

Calculate probability of distance for d-dimensional normal

Is there any simple way to calculate the probability of distance in the following form for d-dimensional normal distribution? $P(||\mathbf{x}-\mathbf{\mu}||^2>||\mathbf{x}-\mathbf{a}||^2)$, where $...