Measures of closeness between distributions, clusterings, data sets or other objects.

learn more… | top users | synonyms

0
votes
2answers
26 views

similarity between two sentences

I have run an experiment where users were given a distorted sentence and asked to restore it. Currently, I am interested to grade the results on the scale between 0 to 1. I'm wondering if there any ...
-1
votes
1answer
14 views

Similarity measure for observations with many variable types

I'd like to do some exploratory data analysis using a 2d plot to demonstrate possible clustering of the subjects in a study. The subjects have many variables recorded, both quantitative and ...
1
vote
0answers
31 views

Using similarity measures as distances

I have a matrix where each row corresponds to an observation with binary attributes, and I am interested in performing multidimensional scaling using cmdscale on ...
0
votes
0answers
9 views

Is there any bibliography to cite the non-metric properties of the Gower distance?

I can't find a paper confirming that Gower distance does not cover all four properties of a metric distance. Any suggestion?
2
votes
0answers
26 views

Covariance function with circular similarity property

My aim is to fit functions to covariance matrices. Furthermore I would like to have these functions positive definite. For example the figure below shows a fitted covariance matrix modeled using a ...
0
votes
0answers
5 views

Calculating “consistency” of variables in a dataset

I know am sorry in advance for the layman question :) I will try to demonstrate by an example: I have a dataset of items in a clothes store with the (dummy) variables: ...
0
votes
0answers
14 views

Binary Classification of Labels with Similar Feature Distributions

I wish to classify gene interactions as 'Validated' and 'Unvalidated' based on certain features of each interaction. Each interaction has 10 different features. However, the feature distributions of ...
0
votes
1answer
55 views

topic similarity semantic PMI between two words wikipedia

I am trying to compute pointwise mutual information (PMI) using wikipedia as data source. Given two words, PMI defines the relation between two words. The formula is as below. ...
1
vote
0answers
17 views

Correlation with stratified data structure

I have a dataset with two stratas, the first level is called block, which represents the census block. the second level is two predetermined subgroups in each blocks, each subgroup contains several ...
1
vote
0answers
34 views

Finding patterns within inconsistent data

If I have a database full of data sets that I collected from a material sample going through light sensors. Each sample is similar but the data can be inconsistent depending on many factors. I ...
2
votes
3answers
55 views

Selecting the most similar subset from an alternative dataset

Background: I have two different datasets coming from two different source. Dataset A has m features (e.g. v1,v2,v3,v4) with let us say 1 million instances. Second dataset B has n features some of ...
0
votes
1answer
37 views

How to quantify the similarity/difference between groups of selected features?

Since I have a imbalanced data to deal with, I randomly down-sample the negative data to reach balanced data for following feature selection and classification. To test feature's robustness to ...
2
votes
3answers
30 views

Cluster similarity matrix as energies

I have a symmetric score matrix and I would like to cluster the values in two dimensions rather than through a tree. Is there any method / library that would take the input matrix and treat the scores ...
0
votes
1answer
30 views

(Non-parametric) test for similarity between three or more vectors of observations?

I have three vectors each of six observations. I imagine other situations will arrive in which I might have more than three vectors with other lengths. I want a statistical test to compare them. ...
0
votes
1answer
41 views

Similarity between two small datasets

I have gone through several test's (Kolmogorov-Smirnov, t-test, u-test etc..) but I am not sure which one to use when it comes to comparing two small data sets, lets say of size 10 each. ...
1
vote
0answers
10 views

Testing if two rankings are equivalent

I have some process A that assigns values y_i for i=1:N to instances x_i. ...
1
vote
0answers
41 views

What are some applications of the Apriori algorithm?

My understanding is that for online promotions the Apriori algorithm is not used as much as say collaborative filtering algorithms largely because the cost of marketing to the long tail is so cheap ...
0
votes
0answers
17 views

Modify dot product for dimension independence

The distribution of the pair-wise dot products of uniformly distributed points on the unit sphere depends upon the dimension n (see, e.g. here). What alternative similarity measures exist that do not ...
1
vote
0answers
16 views

Fisher kernel or KL divergence?

I am currently reading about pLSA and LDA and how can I apply these methods on calculating document similarity. I got a feeling that common similarity measure used in pLSA is Fisher kernel, but for ...
0
votes
0answers
19 views

Diversity Index for binomial data

I have community ecology data where each sample unit has species lists for each of N subsamples. So rather than a raw count of the number of individuals of each species type at each sample unit, I ...
2
votes
1answer
44 views

A similarity measure with binary data: does this one have a name?

There are many binary similarity measures (e.g. Jaccard, Sorensen, etc), each of them is sensitive to different properties of the compared sets. I would like to use the metric $S=\frac{N_{A\bigcap ...
0
votes
0answers
14 views

What is the best practice to deal with NA values when calculating a dissimilarity matrix?

I need to calculate a matrix of distances between sites where different variables were measured. I will use it in a cluster analysis. The following is a sample of the matrix I am dealing with: ...
1
vote
1answer
56 views

Proving similarity between two rankings

We are seeking a comparison of two different rankings of the same 3 systems Background: We have conducted a test of 3 different systems. Participants experienced a system and rated it on a 7-point ...
1
vote
2answers
112 views

What is the best algorithm to find similar text documents?

I have many text documents and I would like to find similar documents to each document within my data set. Is Latent Dirichlet Allocation (LDA) the best way to do that, or are there other algorithms ...
1
vote
1answer
68 views

Comparing undirected weighted graphs

I want to get a sense of the similarity between weighted, undirected graphs. My data are from an EEG experiment, where every vertex is an electrode, and every edge is the connectivity between two ...
0
votes
1answer
51 views

Which (dis)similarity index to choose for cluster analysis?

I have data that refer to the number of occurrences of specific variable in samples: ...
1
vote
0answers
21 views

Similarity Measure for small strings

I am looking for a good similarity measure to conflate entries of a column (Product Brand in my scenario) Text like : ("Dell", "Dell Laptops"), ("ACP","ACP by XYZ"),("Acer Notebooks","Acer ...
0
votes
1answer
21 views

Is there a distance algorithm similar to Jaccard distance that handles scalar data?

we have the characteristics and (scalar) values of those characteristics for three (or more) people: ...
0
votes
0answers
12 views

Similarity of linear subgraphs

I have a directed graph, on which I have a number of linear subgraphs. In the simple example graph below, these subgraph might be a->b->c->e->g->h, ...
1
vote
0answers
14 views

Feature Extraction for landscape images

I work on a landscape-image database (forests, beaches, landmarks, cities etc.) and I'm trying to find the similarity between them, I don't have labels or any specific classes, I just want to find ...
2
votes
1answer
93 views

Overlap between two normal pdfs [duplicate]

I have two normally distributed random variables (estimated from two different sets of samples), and I'd like to know how "similar" those variables are (in order to compare the sets). I had the idea ...
5
votes
2answers
69 views

How to find weights for a dissimiliarity measure

I want to learn (deduce) attribute weights for my dissimilarity measure that I can use for clustering. I have some examples $(a_i,b_i)$ of pairs of objects that are "similar" (should be in the same ...
0
votes
0answers
22 views

Detect rhymes in lyrics

I am testing several features computed from a dataset with several machine learning algorithms, and I would like to see if rhymes contribute to the genre classification of lyrics. Therefore, I would ...
2
votes
1answer
48 views

Turn a distance measure into a kernel function

I have read here that an easy way to turn a distance function $d$ into a similarity function $s$ is to compute: $s = e^{-\gamma * d}$. I believe that this is also what is done with the RBF kernel. ...
1
vote
0answers
12 views

Similarity/duplicate measure for collection of vectors

I have 12 vectors of the size 1x16, which are generated as a side-product of my algorithm. If any of the vectors are very similar, that could indicate that my algorithm is performing badly. Roughly ...
2
votes
1answer
39 views

How to measure similarity of two rectangles' ratios? [closed]

I have two rectangles $R_1=(w_1,h_1)$ and $R_2=(w_2,h_2)$ and I would like to check how similar their ratios are. My first naive thought was this: ...
0
votes
0answers
42 views

Similarity among multivariate time series vs similarity among univariate time series

I am trying to generalize the similarity among univariate time series into multivariate time series. Let's say there are 2 multivariate time series data X and Y: X1, X2, X3 in first multivariate ...
0
votes
0answers
23 views

Using PERMANOVA or BIOENV or both?

I just came across these two methods for analyzing community data and their response to environmental gradients: PERMANOVA (adonis function, package vegan) and bioenv (vegan). As far as I understood, ...
0
votes
0answers
14 views

Determining similarity among 4 datasets based on demographic composition

I have 4 data sets and I am trying to determine what is the order of similarity between Set XY and Sets A, B, or C based on audience characteristics (in other words, which site has the the most ...
6
votes
3answers
320 views

Quantifying similarity between two data sets

Summary: Trying to find the best method summarize the similarity between two aligned data sets of data using a single value. Details: My question is best explained with a diagram. The graphs below ...
4
votes
1answer
98 views

Get k most diverse objects from dendrogram (hierarchical clustering)

I have a dendrogram which groups similar object in a hierarchical order. The problem I try to solve is based on a dendrogram how to get k most diverse objects. E.g. We start with some random (?) ...
0
votes
0answers
30 views

similarities between two signal using correlation

let say we have two signal ,there is given their time domain plots and also correlation plot and final correlation plot it should be noted that both signal have same spectral structure,they ...
0
votes
2answers
428 views

TF-IDF versus Cosine Similarity in Document Search

I'm wondering if anyone can help me out or point out some resources to learn more about TF-IDF and document search. I'm trying to implement a basic document search and am trying to better understand ...
0
votes
0answers
37 views

A good metric for measuring the “similarity” between a number and a series of numbers of unknown distribution

I'm very new to stats so please excuse me if some of my terminologies are wrong. I have a series of N numbers, and I don't know the distribution of these numbers. Now, I have another number K, and I ...
0
votes
0answers
31 views

Correlation/similarity binary time series

Which are the best or most used methods to find similarities between binary time series? ...
3
votes
2answers
102 views

How to measure similarity of bivariate probability distributions?

I have three different distributions of 2D data: or Now I like to know whether distribution two is more similar to distribution one (2 to 1) than distribution three is to distribution one (3 to ...
1
vote
0answers
45 views

recommender - recommendation reasoning (get similarities )

I’m using recommenderlab to build a UBCF and produce recommendations. The process seems to be ok, and the predictions are making sense. Now,what I need to have, is the reasoning for each prediction, ...
1
vote
0answers
68 views

Similarity metric for 2 sets of vectors

I'm trying to determine the similarity between two sentences. I have vectors for each word in a corpus, and using cosine distance of the two vectors, I can get quite a good "similarity" score ...
2
votes
1answer
43 views

Formulation of null and alternative to show two sets have similar mean

I have two sets of ratings between 1 to 5. The second set is generated using an algorithm, and I am currently trying to find the best set of parameters, so the ratings are similar to my first set. ...
3
votes
0answers
20 views

Measuring “almost the same” when significantly different

The following table is of UK domiciled university applicants to UK universities in $2013$ and of acceptances, grouped by self-declared disability. A $\chi^2$ test on the numbers shows an extremely ...