Measures of closeness between distributions, clusterings, data sets or other objects.

learn more… | top users | synonyms

1
vote
1answer
42 views

Proving similarity between two rankings

We are seeking a comparison of two different rankings of the same 3 systems Background: We have conducted a test of 3 different systems. Participants experienced a system and rated it on a 7-point ...
0
votes
2answers
42 views

What is the best algorithm to find similar text documents?

I have many text documents and I would like to find similar documents to each document within my data set. Is Latent Dirichlet Allocation (LDA) the best way to do that, or are there other algorithms ...
1
vote
1answer
41 views

Comparing undirected weighted graphs

I want to get a sense of the similarity between weighted, undirected graphs. My data are from an EEG experiment, where every vertex is an electrode, and every edge is the connectivity between two ...
0
votes
1answer
38 views

Which (dis)similarity index to choose for cluster analysis?

I have data that refer to the number of occurrences of specific variable in samples: ...
2
votes
0answers
18 views

Similarity Measure for small strings

I am looking for a good similarity measure to conflate entries of a column (Product Brand in my scenario) Text like : ("Dell", "Dell Laptops"), ("ACP","ACP by XYZ"),("Acer Notebooks","Acer ...
0
votes
1answer
11 views

Is there a distance algorithm similar to Jaccard distance that handles scalar data?

we have the characteristics and (scalar) values of those characteristics for three (or more) people: ...
0
votes
0answers
11 views

Similarity of linear subgraphs

I have a directed graph, on which I have a number of linear subgraphs. In the simple example graph below, these subgraph might be a->b->c->e->g->h, ...
1
vote
0answers
11 views

Feature Extraction for landscape images

I work on a landscape-image database (forests, beaches, landmarks, cities etc.) and I'm trying to find the similarity between them, I don't have labels or any specific classes, I just want to find ...
2
votes
1answer
75 views

Overlap between two normal pdfs [duplicate]

I have two normally distributed random variables (estimated from two different sets of samples), and I'd like to know how "similar" those variables are (in order to compare the sets). I had the idea ...
5
votes
2answers
63 views

How to find weights for a dissimiliarity measure

I want to learn (deduce) attribute weights for my dissimilarity measure that I can use for clustering. I have some examples $(a_i,b_i)$ of pairs of objects that are "similar" (should be in the same ...
0
votes
0answers
20 views

Detect rhymes in lyrics

I am testing several features computed from a dataset with several machine learning algorithms, and I would like to see if rhymes contribute to the genre classification of lyrics. Therefore, I would ...
2
votes
1answer
31 views

Turn a distance measure into a kernel function

I have read here that an easy way to turn a distance function $d$ into a similarity function $s$ is to compute: $s = e^{-\gamma * d}$. I believe that this is also what is done with the RBF kernel. ...
1
vote
0answers
9 views

Similarity/duplicate measure for collection of vectors

I have 12 vectors of the size 1x16, which are generated as a side-product of my algorithm. If any of the vectors are very similar, that could indicate that my algorithm is performing badly. Roughly ...
2
votes
1answer
23 views

How to measure similarity of two rectangles' ratios? [closed]

I have two rectangles $R_1=(w_1,h_1)$ and $R_2=(w_2,h_2)$ and I would like to check how similar their ratios are. My first naive thought was this: ...
0
votes
0answers
26 views

Similarity among multivariate time series vs similarity among univariate time series

I am trying to generalize the similarity among univariate time series into multivariate time series. Let's say there are 2 multivariate time series data X and Y: X1, X2, X3 in first multivariate ...
0
votes
0answers
16 views

Using PERMANOVA or BIOENV or both?

I just came across these two methods for analyzing community data and their response to environmental gradients: PERMANOVA (adonis function, package vegan) and bioenv (vegan). As far as I understood, ...
0
votes
0answers
11 views

Determining similarity among 4 datasets based on demographic composition

I have 4 data sets and I am trying to determine what is the order of similarity between Set XY and Sets A, B, or C based on audience characteristics (in other words, which site has the the most ...
5
votes
3answers
94 views

Quantifying similarity between two data sets

Summary: Trying to find the best method summarize the similarity between two aligned data sets of data using a single value. Details: My question is best explained with a diagram. The graphs below ...
4
votes
1answer
94 views

Get k most diverse objects from dendrogram (hierarchical clustering)

I have a dendrogram which groups similar object in a hierarchical order. The problem I try to solve is based on a dendrogram how to get k most diverse objects. E.g. We start with some random (?) ...
0
votes
0answers
16 views

similarities between two signal using correlation

let say we have two signal ,there is given their time domain plots and also correlation plot and final correlation plot it should be noted that both signal have same spectral structure,they ...
0
votes
2answers
149 views

TF-IDF versus Cosine Similarity in Document Search

I'm wondering if anyone can help me out or point out some resources to learn more about TF-IDF and document search. I'm trying to implement a basic document search and am trying to better understand ...
0
votes
0answers
18 views

A good metric for measuring the “similarity” between a number and a series of numbers of unknown distribution

I'm very new to stats so please excuse me if some of my terminologies are wrong. I have a series of N numbers, and I don't know the distribution of these numbers. Now, I have another number K, and I ...
0
votes
0answers
23 views

Correlation/similarity binary time series

Which are the best or most used methods to find similarities between binary time series? ...
3
votes
2answers
93 views

How to measure similarity of bivariate probability distributions?

I have three different distributions of 2D data: or Now I like to know whether distribution two is more similar to distribution one (2 to 1) than distribution three is to distribution one (3 to ...
0
votes
0answers
34 views

recommender - recommendation reasoning (get similarities )

I’m using recommenderlab to build a UBCF and produce recommendations. The process seems to be ok, and the predictions are making sense. Now,what I need to have, is the reasoning for each prediction, ...
1
vote
0answers
60 views

Similarity metric for 2 sets of vectors

I'm trying to determine the similarity between two sentences. I have vectors for each word in a corpus, and using cosine distance of the two vectors, I can get quite a good "similarity" score ...
2
votes
1answer
42 views

Formulation of null and alternative to show two sets have similar mean

I have two sets of ratings between 1 to 5. The second set is generated using an algorithm, and I am currently trying to find the best set of parameters, so the ratings are similar to my first set. ...
3
votes
0answers
19 views

Measuring “almost the same” when significantly different

The following table is of UK domiciled university applicants to UK universities in $2013$ and of acceptances, grouped by self-declared disability. A $\chi^2$ test on the numbers shows an extremely ...
0
votes
0answers
31 views

how to measure the similarity or overlapping significance between two character vectors with pre-ranked items?

Here there are two character vectors (like V1 and V2). Items in each vector are unique, which means no duplicate in each vector. All the items in the vectors come from a known set (#item in set >> ...
0
votes
1answer
34 views

MinHashing and Hash Digests/Signatures

Nilsimsa and SimHash are two similarity hash algorithms that produce a single hash digest/signature per document. Comparing two documents is as simple as computing the hamming distance between two ...
2
votes
1answer
46 views

Compare two unnormalized density functions given the values at samples

I have the density values of two unnormalized density functions $p$ and $q$ at 2000 points: $\mathbf{p} = (p(x_1), p(x_2), \dots p(x_{2000}))$, $\mathbf{q} = (q(x_1), q(x_2), \dots q(x_{2000}))$. Now ...
0
votes
1answer
59 views

Similarity scoring to compare multi-dimensional datasets

I am trying to come up with a mechanism of scoring a set of multidimensional datasets based on a similarity with an ideal dataset. Each dataset will all have the same dimensions along with the ideal. ...
0
votes
0answers
104 views

How to measure consistency or simiarity between two data sets?

I am working on a study that elicits priorities (weights) from experts on a set of alternatives using different methods. The snapshot below shows the resulting weights, which sum up to 1 for each ...
1
vote
0answers
13 views

Similarity measure of lower and upper triangle matrix [closed]

Do you have any idea to create index regarding the similarity of lower and upper elements (i-jth element and j-i th element) of triangle matrix A? One way is given the index as Tij = A[ij] / A[ji] ...
1
vote
0answers
33 views

Similarity measure for time series

How can I recognize patterns with different sizes in a time-series? Imagine that I have a template pattern and I need to find that pattern in a symbolic representation of the time-series (stock market ...
3
votes
1answer
83 views

A valid distance metric for high dimensional data

I asked a question about forming a valid distance metric yesterday (Link1) and got some very good answer; however, I have got some more questions about forming a proper distance metric for high ...
1
vote
3answers
100 views

How to get a valid distance metric?

I have got a problem to devise a distance metric to get the similarity measurement of vectors. Someone suggested me to use dot product, which seems to me the same as the Cosine similarity metric; ...
0
votes
0answers
21 views

Correlation or similarity between response surfaces

Given the dataset A and B both containing observations (x, y) whereas x can be a vector. What are some possibilities of measuring the similarity or correlation between A and B? We can assume that ...
7
votes
2answers
142 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
3
votes
2answers
81 views

Is paired t test sufficient to show the two data sets are similar

Let me give the details of my problem setting, we have a learning algorithm that can take a collection of algebra problems and automatically generate similar ones. Of course, an important question to ...
0
votes
0answers
69 views

How to quantify the similarity between two samples using quartiles only?

I have several sets of samples I would like to compare. Each set is comprised of two samples, for which I only have the quartiles, min/max values and sample size for each sample. I would like to ...
1
vote
1answer
136 views

How to superimpose two MATLAB images rigidly transformed to perform a metrics

I got two MRI images on MATLAB, I need to perform an intensity based registration similarity metrics in order to get a registration. The problem is that, since I got some rigid transformation on one ...
0
votes
0answers
110 views

Two step cluster analysis and a binary matching coefficient

I want to commence a two-step cluster analysis, since the database I am conducting analysis on contains important metric as well as nominal values. => Question #1: Should the binary and the metric ...
0
votes
0answers
68 views

How to use Similarity Measure in K-nearest neighbor Classification?

I have a similarity measure just like cosine. How can i use that similarity measure in traditional k-nn classification? Please provide some literature review (research papers) details which i should ...
2
votes
1answer
61 views

Drop highly correlated items?

I have two IVs that are highly correlated with each other at 0.979 (Pearson) & 0.919 (Kendall's). IV1: Quality of response IV2: Quality of Technical Advice Sample Size: 252 Considering the ...
0
votes
0answers
112 views

Using relative frequency for Euclidean and cosine distance (dissimilarity)

How to calculate the Euclidean distance (dissimilarity) between two documents, e.g., D1 and D2 using relative frequency? Here is an example of both cosine and Euclidean distance between two ...
0
votes
0answers
161 views

Calculate Similarities between two arrays with different scales

In creating a content-based recommender, I have several binary measurements, combined with integer measurement values and I wish to compare the two vectors. For example, consider the following two ...
0
votes
1answer
320 views

kernels and similarity (in R)

I am trying fit different kernels to calculate similarity matrix in R. Here is example data - X matrix : ...
0
votes
0answers
37 views

Comparing similarity of two fingerprint or bitvector

I have a question that's two part. 1) The first part is that I have a bit vector or a fingerprint of a digital circuit where the each bit is high if a specific structure or component exists and zero ...
0
votes
0answers
15 views

Search for similar documents on growing term-base

Could you please recommend any approach (if it exists) for similar document search considering the following: It's necessary to provide term base explicitly. This is a dictionary of specific named ...