# Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

504 questions
Filter by
Sorted by
Tagged with
14 views

### A common mathematical framework to introduce different "measures of distributional divergences/similarity"

Preface. I am going to ask here the same question ("A common mathematical framework to introduce different "measures of distributional divergences/similarity") that I asked on math....
16 views

### How to test similarity between multiple medical differential diagnoses?

I have multiple observations collected by doctors where they give the differential diagnoses, the 10 most probable diseases (d). I want to calculate the inter-observer similarity. my data looks like ...
31 views

### Calculate the cosine similarity between two probability measures, rather then between two vectors

Introduction. The cosine similarity is often calculated among two vectors, i.e. $C_S(X,Y)$: \begin{equation} C_S(X,Y) = \frac{\sum_{i=1}^{n}(X_i,Y_i)}{\sqrt{\sum_{i=1}^{n}X_{i}^{2} \sum_{i=1}^{n}Y_{i}^...
13 views

### How to adjust similarity scores by removing the influence of a common vector?

I have a similarity score function, $s(x,y)$. I know that I have two items that I'm trying to compare the similarity of, but both are based on the same template. How would I remove the template from ...
15 views

### Timeline "spike" similarity metric?

Imagine you have a single axis. Along the axis are placed several target points, at differing distances between each other. You now have a black box that shoots a variable number of points on to the ...
17 views

### Specific correlation metric for categorical variables where matched values are of primary interest - does it have a name?

Hello fellow StackExchangers, Let's say I have an experiment where two people are asked several questions, and for each question, they can independently select one of 5 preformed answers (A to E). So ...
1 vote
9 views

### Similarity metrics for circular and unaligned time series

I face the problem of calculating the similarity between multiple time series. What I want to do is compare each one with the first one to quantify the degree of change. The challenge is that they are ...
1 vote
50 views

### Normalizing Euclidean distance by the length of the vectors [closed]

Suppose I have 4 vectors, the first 2 vectors are of length 4 and the last 2 vectors are of length 400. all values in the vectors range from 0.5 to 0.6. The Euclidean distance between the last 2 ...
47 views

### Correlation vs Euclidean distance as measures of similarity or closeness between data points with an outlier

I am interested in the comparison of Pearson correlation and Euclidean distance as measures of similarity between data points. Suppose I have 4 data points, w, x, y, z, in a multidimensional space, ...
46 views

### Apply divergences between two "relative frequency distributions", instead of between two "probability distributions"

Introduction. Recalling that: The frequency is the number of observations of a specific outcome. The relative frequency is a proportion of all observations (frequency / total observations). A "...
35 views

### Weighted entropy using similarity scores

I have a list (freq) of letters with the following frequencies: ...
11 views

### Differential Privacy guarantee that takes into account the approximate density (e.g., the pseudo randomness) used in practice?

In theory the differential privacy guarantee comes from adding randomness to an algorithm so whatever is output is a sample from a target distribution (e.g., the Laplacian, Gaussian, Exponential ...
32 views

### Does the number of bins influence the "measures of similarity" among histograms?

I have calculated several "measures of similarity" among histograms and discrete probability distributions (such as Cosine similarity, Kullback-Leibler divergence, Jensen-Shannon distance, ...
21 views

### How to normalise dissimilarity and exposure indices (for longitudinal data)

I'm currently working with longitudinal data and I wanted to compare school and residential segregation (natives vs minorities) over time and across specific territorial sub-units (NUTS 3 level). The ...
96 views

### Sample size/ power estimation for similarity index (Dice/ F1 Score)

We will start a study in order to validate an algorithm for image segmentation for medical use. We will use Dice Sorensen coefficient in order to measure the agreement/ similarity between the ...
71 views

### Finding trend similarity between two different time series

I am very new to this data analytics field, so bit confused about what to do. I have 2 time series datasets of two different products that have the same attributes. I want to find trend similarity ...
31 views

### Measuring similarity between two datasets

This is quite straightforward: I have a dataset with 8 features containing cell measurements from a patient. I want to compare two patients to a reference one. To do this I perform UMAP to reduce the ...
53 views

### Should similarity matrix must always be converted into a dissimilarity matrix for hierarchical clustering?

Background Hi all, I need some clarification on my approach if it's correct or not. I have a matrix (M_ij) with user ratings of images. The users (i) are on the horizontal axis and the images (j) are ...
1 vote
10 views

### How to include a nested factor in a SIMPER test in PRIMER [closed]

I'm trying to conduct a SIMPER analysis in PRIMER. I'm wanting to do the analysis using a factor that is made up of the interaction between 2 other factors, with one nested in a third. I'm looking at ...
76 views

### How to compare two dendrograms

I want to find a method to compare two dendrograms. I have documents that were read by some people and they made a dendrogram that indicates which documents they think are more similar. Now I want to ...
12 views

### Similarity measure over time

I'm completely new to this so I apologise in advance if my question isn't even phrased correctly. I want to analyse changes in similarity of two interacting entities over time. Each entity is ...
20 views

### Does this metric exist for clustering data based off its similarity matrix?

I'm working with a dataset where all I have is the similarity matrix (with values 0 to 1, 0 being no similarity, 1 being identical). After I assign the labels, I loop through the matrix and, for each ...
18 views

### Similarity of two sets of scatter data

I am currently working on a project where I have a dataset of scatter-points in space. What I am doing is that I fit Gaussian Mixture Models (GMM) on the data set and use the model to sample '...
1 vote
16 views

### Measure "similarity" of different feature sets on the same samples

I have a population that I have quantitatively profiled using two different technologies. The result is two data matrices that have the exact same set of individuals as rows, but different features (...
1 vote
45 views

### Finding the nearest point to a given set of points [closed]

I am trying to build an auto-suggestion logic which looks at currently selected items and recommends a list of items to select the next item from. I can formulate the problem thus: Given a set of ...
114 views

### How to measure the similarity of frequency of categorical data

Sorry if this is a really simple question but I'm new to this and wondered if there's an easy way to do what I'm picturing. Imagine I've got a bunch of people and I'm asking them what they've eaten in ...
17 views

### Matching items based on similarity

I would like to ask for a proposal for a machine learning model that would be suitable for the following problem: I have a training set where each element of type A corresponds to a certain number of ...
35 views

### Compare similarity or difference of two distributions by the ratio of moments

I'm looking for a measure that measure the similarity of two distributions in the following forms: $$S = \frac{a \mu_1 + b \sigma_1}{a \mu_2 + b \sigma_2}.$$ The above formula I proposed is not ...
1 vote
42 views

### Similarity measure for two discrete distributions (a group of proportions)

I am not really sure how to describe it with the correct mathematical terms. If there are any questions, I will try to explain further. I have several instances of a group of proportions. One could ...
1 vote
24 views

### Dimensionality reduction that preserves non-trivial similarity

I have a set of vectors $x_i \in\mathbb{R}^m$ and a similarity function f that quantifies how similar $x_i,x_j$ are. Unfortunately, calculating f takes a lot of time. I want to use some neural ...
91 views

### How to calculate the Jaccard index of co-occurence matrix without diagonal values?

I transformed a 2-mode matrix into a 1-mode co-occurence matrix and canceled all the values on the diagonal (because I thought it was meaningless). I wanted to use the jaccard method to calculate the ...
35 views

### Comparing the similarity of ranked word lists with a reference standard

Imagine a scientific organisation wants to communicate the likelihood of particular events occurring. For consistency, the organisation decides to always draw from the same set of words, which it ...
1 vote
52 views

### R adjusted cosine similarity [closed]

I would like to find an effective way to get adjusted cosine similarity for a large matrix (10k rows). Apply seems also slow... Could you please suggest a method? Maybe based on the approaches ...
39 views

### Calculating similarity/resemblance of time series data? (soil moisture)

I have measured a variable (soil moisture) for multiple samples over time. Soil moisture increases steeply at several points in time (when it rains) and then decreases steadily until the next rain ...
1 vote
35 views

1 vote
28 views

### The probability of two measurements being identical when both have measurement errors

Statement: Let's assume $X_i =\{x_1,\ x_2,\ldots,\ x_i \}$ is a set of products that I am interested in their length. I have two measurement devices that measure the length of these products in ...
38 views

### Compare discrete time series on equalization

I have some discrete time series like this: ...
1 vote
93 views

### Find similarity of two ranked lists that have weights

Suppose I have two lists with columns rank, app_name, weight that look like this: ...
23 views

### How can I discover similar or resembling sets by a statistical method?

Let say, I have such m datasets: ...
1 vote
46 views

### How to define a distance metric between nodes of an Origin-Destination matrix?

I have an Origin-Destination matrix expressing (weekly) flows of people between every couple of nodes (cities). The number of people traveling from city $i$ to city $j$ in a specific week is $OD_{ij}$....
67 views

### How best to compare two processes to know if they perform statistically the same?

I have two processes, A and B. I have tested A and B separately on different tasks, and each performed at a 90% level on their tasks (say, text classification). Now, I pick a random set of 100 tasks ...
95 views

### Similarity percentage based on attributes and their quantities

I have two sets that contain (partially overlapping) attributes in different quantities and I am looking for a method to compare the similarity between the sets based on the attributes and their ...
363 views

### Is there any well-founded way of calculating the euclidean distance between two images?

I need to determine the distance between two images. Supposing that we are dealing with images of the same size, I think that we can reduce this problem to the square root of the sum of the square of ...