Measures of closeness between distributions, clusterings, data sets or other objects.

learn more… | top users | synonyms

0
votes
1answer
31 views

Quantification of the extent of periodicity in a time series using fractal analyses

I need metrics to quantify the extent of periodicity between of a time series (for comparison with other time series), considering the time series is almost periodic. By almost periodic I mean: if I ...
0
votes
0answers
15 views

performing a partial correlation using cosine similarity

I wonder if it's possible to carry out a partial correlation using correlation measures other than the Pearson or Spearman correlation. In particular, I want to assess the cosine similarity between X ...
1
vote
0answers
18 views

Measure how different are two groups in their answering on 25 ordinal variables

The research context: I'm interested in usage frequency of special interest websites and its predictors. I asked individuals (n=1000) for the usage of 25 different websites in 25 items in a survey ...
0
votes
1answer
23 views

Composite similarity of GitHub projects

As a hobby, I'm building recommendation system which finds related projects on GitHub. I'm computing Jaccard index for each repository, based on users who gave stars: $$J(A, B) = \frac{ \left\vert ...
0
votes
0answers
10 views

Graph Edit Distance - strings versus matrix-->vector conversion?

I have two adjacency matrices and I want to find the graph edit distance between them. The only implementations I'm seeing (I need this fast) is edit distances in MATLAB (sigh...) for strings. I have ...
1
vote
0answers
40 views

Verify model correctness

I try to explain my question with an example. Assume we have a network of 3 computers. We can measure different entity in this system such as CPU load, disk load, Log messages and etc. (all the ...
0
votes
2answers
33 views

similarity between two sentences

I have run an experiment where users were given a distorted sentence and asked to restore it. Currently, I am interested to grade the results on the scale between 0 to 1. I'm wondering if there any ...
-1
votes
1answer
20 views

Similarity measure for observations with many variable types

I'd like to do some exploratory data analysis using a 2d plot to demonstrate possible clustering of the subjects in a study. The subjects have many variables recorded, both quantitative and ...
1
vote
0answers
41 views

Using similarity measures as distances

I have a matrix where each row corresponds to an observation with binary attributes, and I am interested in performing multidimensional scaling using cmdscale on ...
0
votes
0answers
10 views

Is there any bibliography to cite the non-metric properties of the Gower distance?

I can't find a paper confirming that Gower distance does not cover all four properties of a metric distance. Any suggestion?
2
votes
0answers
33 views

Covariance function with circular similarity property

My aim is to fit functions to covariance matrices. Furthermore I would like to have these functions positive definite. For example the figure below shows a fitted covariance matrix modeled using a ...
0
votes
0answers
9 views

Calculating “consistency” of variables in a dataset

I know am sorry in advance for the layman question :) I will try to demonstrate by an example: I have a dataset of items in a clothes store with the (dummy) variables: ...
0
votes
0answers
15 views

Binary Classification of Labels with Similar Feature Distributions

I wish to classify gene interactions as 'Validated' and 'Unvalidated' based on certain features of each interaction. Each interaction has 10 different features. However, the feature distributions of ...
0
votes
1answer
79 views

topic similarity semantic PMI between two words wikipedia

I am trying to compute pointwise mutual information (PMI) using wikipedia as data source. Given two words, PMI defines the relation between two words. The formula is as below. ...
1
vote
0answers
22 views

Correlation with stratified data structure

I have a dataset with two stratas, the first level is called block, which represents the census block. the second level is two predetermined subgroups in each blocks, each subgroup contains several ...
1
vote
0answers
35 views

Finding patterns within inconsistent data

If I have a database full of data sets that I collected from a material sample going through light sensors. Each sample is similar but the data can be inconsistent depending on many factors. I ...
2
votes
3answers
55 views

Selecting the most similar subset from an alternative dataset

Background: I have two different datasets coming from two different source. Dataset A has m features (e.g. v1,v2,v3,v4) with let us say 1 million instances. Second dataset B has n features some of ...
0
votes
1answer
42 views

How to quantify the similarity/difference between groups of selected features?

Since I have a imbalanced data to deal with, I randomly down-sample the negative data to reach balanced data for following feature selection and classification. To test feature's robustness to ...
2
votes
3answers
34 views

Cluster similarity matrix as energies

I have a symmetric score matrix and I would like to cluster the values in two dimensions rather than through a tree. Is there any method / library that would take the input matrix and treat the scores ...
0
votes
1answer
39 views

(Non-parametric) test for similarity between three or more vectors of observations?

I have three vectors each of six observations. I imagine other situations will arrive in which I might have more than three vectors with other lengths. I want a statistical test to compare them. ...
0
votes
1answer
44 views

Similarity between two small datasets

I have gone through several test's (Kolmogorov-Smirnov, t-test, u-test etc..) but I am not sure which one to use when it comes to comparing two small data sets, lets say of size 10 each. ...
1
vote
0answers
12 views

Testing if two rankings are equivalent

I have some process A that assigns values y_i for i=1:N to instances x_i. ...
1
vote
0answers
49 views

What are some applications of the Apriori algorithm?

My understanding is that for online promotions the Apriori algorithm is not used as much as say collaborative filtering algorithms largely because the cost of marketing to the long tail is so cheap ...
0
votes
0answers
19 views

Modify dot product for dimension independence

The distribution of the pair-wise dot products of uniformly distributed points on the unit sphere depends upon the dimension n (see, e.g. here). What alternative similarity measures exist that do not ...
1
vote
0answers
17 views

Fisher kernel or KL divergence?

I am currently reading about pLSA and LDA and how can I apply these methods on calculating document similarity. I got a feeling that common similarity measure used in pLSA is Fisher kernel, but for ...
0
votes
0answers
20 views

Diversity Index for binomial data

I have community ecology data where each sample unit has species lists for each of N subsamples. So rather than a raw count of the number of individuals of each species type at each sample unit, I ...
2
votes
1answer
51 views

A similarity measure with binary data: does this one have a name?

There are many binary similarity measures (e.g. Jaccard, Sorensen, etc), each of them is sensitive to different properties of the compared sets. I would like to use the metric $S=\frac{N_{A\bigcap ...
0
votes
0answers
15 views

What is the best practice to deal with NA values when calculating a dissimilarity matrix?

I need to calculate a matrix of distances between sites where different variables were measured. I will use it in a cluster analysis. The following is a sample of the matrix I am dealing with: ...
1
vote
1answer
61 views

Proving similarity between two rankings

We are seeking a comparison of two different rankings of the same 3 systems Background: We have conducted a test of 3 different systems. Participants experienced a system and rated it on a 7-point ...
1
vote
2answers
132 views

What is the best algorithm to find similar text documents?

I have many text documents and I would like to find similar documents to each document within my data set. Is Latent Dirichlet Allocation (LDA) the best way to do that, or are there other algorithms ...
1
vote
1answer
84 views

Comparing undirected weighted graphs

I want to get a sense of the similarity between weighted, undirected graphs. My data are from an EEG experiment, where every vertex is an electrode, and every edge is the connectivity between two ...
0
votes
1answer
69 views

Which (dis)similarity index to choose for cluster analysis?

I have data that refer to the number of occurrences of specific variable in samples: ...
1
vote
0answers
22 views

Similarity Measure for small strings

I am looking for a good similarity measure to conflate entries of a column (Product Brand in my scenario) Text like : ("Dell", "Dell Laptops"), ("ACP","ACP by XYZ"),("Acer Notebooks","Acer ...
0
votes
1answer
22 views

Is there a distance algorithm similar to Jaccard distance that handles scalar data?

we have the characteristics and (scalar) values of those characteristics for three (or more) people: ...
0
votes
0answers
12 views

Similarity of linear subgraphs

I have a directed graph, on which I have a number of linear subgraphs. In the simple example graph below, these subgraph might be a->b->c->e->g->h, ...
1
vote
0answers
15 views

Feature Extraction for landscape images

I work on a landscape-image database (forests, beaches, landmarks, cities etc.) and I'm trying to find the similarity between them, I don't have labels or any specific classes, I just want to find ...
2
votes
1answer
114 views

Overlap between two normal pdfs [duplicate]

I have two normally distributed random variables (estimated from two different sets of samples), and I'd like to know how "similar" those variables are (in order to compare the sets). I had the idea ...
5
votes
2answers
77 views

How to find weights for a dissimiliarity measure

I want to learn (deduce) attribute weights for my dissimilarity measure that I can use for clustering. I have some examples $(a_i,b_i)$ of pairs of objects that are "similar" (should be in the same ...
0
votes
0answers
22 views

Detect rhymes in lyrics

I am testing several features computed from a dataset with several machine learning algorithms, and I would like to see if rhymes contribute to the genre classification of lyrics. Therefore, I would ...
2
votes
1answer
49 views

Turn a distance measure into a kernel function

I have read here that an easy way to turn a distance function $d$ into a similarity function $s$ is to compute: $s = e^{-\gamma * d}$. I believe that this is also what is done with the RBF kernel. ...
1
vote
0answers
13 views

Similarity/duplicate measure for collection of vectors

I have 12 vectors of the size 1x16, which are generated as a side-product of my algorithm. If any of the vectors are very similar, that could indicate that my algorithm is performing badly. Roughly ...
2
votes
1answer
45 views

How to measure similarity of two rectangles' ratios? [closed]

I have two rectangles $R_1=(w_1,h_1)$ and $R_2=(w_2,h_2)$ and I would like to check how similar their ratios are. My first naive thought was this: ...
0
votes
0answers
45 views

Similarity among multivariate time series vs similarity among univariate time series

I am trying to generalize the similarity among univariate time series into multivariate time series. Let's say there are 2 multivariate time series data X and Y: X1, X2, X3 in first multivariate ...
0
votes
0answers
27 views

Using PERMANOVA or BIOENV or both?

I just came across these two methods for analyzing community data and their response to environmental gradients: PERMANOVA (adonis function, package vegan) and bioenv (vegan). As far as I understood, ...
0
votes
0answers
15 views

Determining similarity among 4 datasets based on demographic composition

I have 4 data sets and I am trying to determine what is the order of similarity between Set XY and Sets A, B, or C based on audience characteristics (in other words, which site has the the most ...
6
votes
3answers
484 views

Quantifying similarity between two data sets

Summary: Trying to find the best method summarize the similarity between two aligned data sets of data using a single value. Details: My question is best explained with a diagram. The graphs below ...
4
votes
1answer
100 views

Get k most diverse objects from dendrogram (hierarchical clustering)

I have a dendrogram which groups similar object in a hierarchical order. The problem I try to solve is based on a dendrogram how to get k most diverse objects. E.g. We start with some random (?) ...
0
votes
0answers
35 views

similarities between two signal using correlation

let say we have two signal ,there is given their time domain plots and also correlation plot and final correlation plot it should be noted that both signal have same spectral structure,they ...
0
votes
2answers
563 views

TF-IDF versus Cosine Similarity in Document Search

I'm wondering if anyone can help me out or point out some resources to learn more about TF-IDF and document search. I'm trying to implement a basic document search and am trying to better understand ...
0
votes
0answers
44 views

A good metric for measuring the “similarity” between a number and a series of numbers of unknown distribution

I'm very new to stats so please excuse me if some of my terminologies are wrong. I have a series of N numbers, and I don't know the distribution of these numbers. Now, I have another number K, and I ...