Skip to main content

Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

Filter by
Sorted by
Tagged with
0 votes
3 answers
66 views

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

I have functions that on iterval [0,1] all seem to look like this: i.e. they have a zero around 0.4 +ve derivative from zero to 0.4 and around zero or slightly negative derivative up to 1. I plan to ...
ufghd34's user avatar
0 votes
0 answers
9 views

Similarity measure for sparse, ordered, binary vectors, with more weighting to True values

I have two sparse, ordered, binary vectors. The size of the vectors is around ~100. I am under the impression that cosine similarity is useful for sparse, ordered, binary vectors. For my purposes, it ...
feonyte's user avatar
  • 69
0 votes
0 answers
17 views

How to compare multiple point patterns of the same realm resulting from different clustering methods?

I have a dataset of 500 sampling points to which I apply multiple different clustering methods for categorization. How can I compare the resulting spatial point patterns of cluster group labels? I ...
jlklein's user avatar
2 votes
0 answers
56 views

Using one distance metric on another distance matrix

In general, is it correct to use one distance metric on another distance matrix? For example, is it valid to use Euclidean metric on Jaccard distance matrix as input data in algorithms? or any other ...
rkabuk's user avatar
  • 71
0 votes
0 answers
10 views

How to convert pairwise dissimilarity matrix into continues predictor in R

I have collected plant species data and AGB from 25 different sites and I want to do regression analysis of Beta-diversity (i.e. dissimilarity among 25 sites) as a predictor and AGB as response ...
Gossaye H's user avatar
4 votes
1 answer
89 views

How can I compare the elements of two lists (in terms of overall similarity and order) in R?

I have a list of the 'actual' top 25 authors and I want to compare this to lists of predicted top 25 authors (I have 5x repeats of the predicted list). I want to compare actual and predicted both in ...
banananada's user avatar
0 votes
0 answers
17 views

Measures of similarity for time series data

I've got two year's worth of energy data in 15 minute increments, and need to develop a similarity score for a forecasted day i.e. identify past days that are similar to the forecasted day. I started ...
user avatar
0 votes
0 answers
10 views

Distance to find similar samples in a multivariate dataset

Apart from Euclidean and Mahalanobis distance metrics. Given a sample with multivariable values, is there a way to find the samples that are similar to the given sample? Does KNN clustering find the ...
sveer's user avatar
  • 103
1 vote
0 answers
19 views

Finding the most-similar color palette [closed]

I have a dataset of several thousand color palettes: for each row in a dataset, I have the ten most common RGB values in an image, and the fraction of that color in the overall image: ...
Saul Aryeh Kohn's user avatar
0 votes
0 answers
10 views

Comparing two datasets of communication flows

I know the question of how to compare two datasets has been asked multiple times and I have read a lot about the different tests available but I'm a little lost on what exactly the tests show and what ...
Milan D's user avatar
  • 101
0 votes
0 answers
24 views

Is it possible to convert a similarity (distance) index to correlation coefficient?

I have two cases that each one has some values on a series of variables (e.g. A, B & C). Is it possible to calculate a distance or similarity index between these two cases and then convert it to a ...
Mahdi Karvandi's user avatar
0 votes
0 answers
68 views

Similarity and Prevalence in Binary Vectors

Let's say I have N vectors, all of length L. Each vector is binary, such that they comprise of 0s and 1s whereby a 0 represents an 'absence' and 1 represents a 'presence' of an element denoted by its ...
raja's user avatar
  • 31
0 votes
0 answers
15 views

Spatial autocorrelation weight matrix for heatmaps

I am not an advanced mathematician and I'm very new to spatial autocorrelation. So I apologize if my question seems primitive in any way. I'm looking for ways to quantify the distribution similarity ...
Saeed Aram's user avatar
6 votes
2 answers
132 views

Is there sampling process that admits computing a similarity of two densities when one is intractable?

I have two densities, $p, q$ with sample space $\mathbb{R}^n$, and we can assume both $p,q>0$ (full support). I can compute and sample from $q$. I can compute $p$ up to a constant and I cannot ...
travelingbones's user avatar
0 votes
0 answers
17 views

Finding similar localities based on age demographics

I'm using the Australian Census data to find similar towns based on their age demographics. Some locations show a large young population, some more working age, some older with a greater number of ...
MrTelly's user avatar
  • 103
3 votes
1 answer
128 views

Appropriate distance measure for comparing probability distributions

Suppose I want to compare some countries to see how similar they are with respect to the relative sizes of the industries in them. So I find data on the distribution of GDP across all industries for ...
user32038's user avatar
0 votes
0 answers
51 views

Determining similarity between two equations

For some work I am doing, I want to determine how similar a set of linear equations are to y=x The other linear equations are determined through linear regression of different datasets (all of which ...
Stephen's user avatar
0 votes
0 answers
136 views

Is there a better alternative for the Jaccard index to obtain the relationship between binary indices?

I currently have data from $n$ sources. Each source sends a binary signal each minute. I have a register of $p$ minutes. I have used Python to calculate the Jaccard indices of each signal as ...
slow_learner's user avatar
0 votes
0 answers
22 views

How to test similarity between multiple medical differential diagnoses?

I have multiple observations collected by doctors where they give the differential diagnoses, the 10 most probable diseases (d). I want to calculate the inter-observer similarity. my data looks like ...
Timothy's user avatar
0 votes
0 answers
58 views

Calculate the cosine similarity between two probability measures, rather then between two vectors

Introduction. The cosine similarity is often calculated among two vectors, i.e. $C_S(X,Y)$: \begin{equation} C_S(X,Y) = \frac{\sum_{i=1}^{n}(X_i,Y_i)}{\sqrt{\sum_{i=1}^{n}X_{i}^{2} \sum_{i=1}^{n}Y_{i}^...
Ommo's user avatar
  • 280
0 votes
0 answers
21 views

How to adjust similarity scores by removing the influence of a common vector?

I have a similarity score function, $s(x,y)$. I know that I have two items that I'm trying to compare the similarity of, but both are based on the same template. How would I remove the template from ...
user760900's user avatar
0 votes
0 answers
16 views

Timeline "spike" similarity metric?

Imagine you have a single axis. Along the axis are placed several target points, at differing distances between each other. You now have a black box that shoots a variable number of points on to the ...
user3735204's user avatar
0 votes
1 answer
21 views

Specific correlation metric for categorical variables where matched values are of primary interest - does it have a name?

Hello fellow StackExchangers, Let's say I have an experiment where two people are asked several questions, and for each question, they can independently select one of 5 preformed answers (A to E). So ...
Jason's user avatar
  • 3
1 vote
0 answers
14 views

Similarity metrics for circular and unaligned time series

I face the problem of calculating the similarity between multiple time series. What I want to do is compare each one with the first one to quantify the degree of change. The challenge is that they are ...
manufacturing_analytics123's user avatar
2 votes
1 answer
302 views

Normalizing Euclidean distance by the length of the vectors [closed]

Suppose I have 4 vectors, the first 2 vectors are of length 4 and the last 2 vectors are of length 400. all values in the vectors range from 0.5 to 0.6. The Euclidean distance between the last 2 ...
user17420392's user avatar
2 votes
2 answers
635 views

Correlation vs Euclidean distance as measures of similarity or closeness between data points with an outlier

I am interested in the comparison of Pearson correlation and Euclidean distance as measures of similarity between data points. Suppose I have 4 data points, w, x, y, z, in a multidimensional space, ...
taellipsis's user avatar
-1 votes
1 answer
58 views

Apply divergences between two "relative frequency distributions", instead of between two "probability distributions"

Introduction. Recalling that: The frequency is the number of observations of a specific outcome. The relative frequency is a proportion of all observations (frequency / total observations). A "...
Ommo's user avatar
  • 280
0 votes
1 answer
63 views

Weighted entropy using similarity scores

I have a list (freq) of letters with the following frequencies: ...
isthisthat's user avatar
2 votes
1 answer
26 views

Differential Privacy guarantee that takes into account the approximate density (e.g., the pseudo randomness) used in practice?

In theory the differential privacy guarantee comes from adding randomness to an algorithm so whatever is output is a sample from a target distribution (e.g., the Laplacian, Gaussian, Exponential ...
travelingbones's user avatar
0 votes
0 answers
76 views

Does the number of bins influence the "measures of similarity" among histograms?

I have calculated several "measures of similarity" among histograms and discrete probability distributions (such as Cosine similarity, Kullback-Leibler divergence, Jensen-Shannon distance, ...
Ommo's user avatar
  • 280
0 votes
1 answer
267 views

Sample size/ power estimation for similarity index (Dice/ F1 Score)

We will start a study in order to validate an algorithm for image segmentation for medical use. We will use Dice Sorensen coefficient in order to measure the agreement/ similarity between the ...
JP1's user avatar
  • 201
0 votes
1 answer
155 views

Finding trend similarity between two different time series

I am very new to this data analytics field, so bit confused about what to do. I have 2 time series datasets of two different products that have the same attributes. I want to find trend similarity ...
Divya Prakash's user avatar
0 votes
0 answers
43 views

Measuring similarity between two datasets

This is quite straightforward: I have a dataset with 8 features containing cell measurements from a patient. I want to compare two patients to a reference one. To do this I perform UMAP to reduce the ...
amr95's user avatar
  • 13
1 vote
0 answers
40 views

How to include a nested factor in a SIMPER test in PRIMER [closed]

I'm trying to conduct a SIMPER analysis in PRIMER. I'm wanting to do the analysis using a factor that is made up of the interaction between 2 other factors, with one nested in a third. I'm looking at ...
Psk's user avatar
  • 11
2 votes
1 answer
161 views

How to compare two dendrograms

I want to find a method to compare two dendrograms. I have documents that were read by some people and they made a dendrogram that indicates which documents they think are more similar. Now I want to ...
stachuk's user avatar
  • 21
0 votes
0 answers
21 views

Does this metric exist for clustering data based off its similarity matrix?

I'm working with a dataset where all I have is the similarity matrix (with values 0 to 1, 0 being no similarity, 1 being identical). After I assign the labels, I loop through the matrix and, for each ...
user386237's user avatar
0 votes
0 answers
22 views

Similarity of two sets of scatter data

I am currently working on a project where I have a dataset of scatter-points in space. What I am doing is that I fit Gaussian Mixture Models (GMM) on the data set and use the model to sample '...
Kelvin Walenta's user avatar
1 vote
0 answers
37 views

Measure "similarity" of different feature sets on the same samples

I have a population that I have quantitatively profiled using two different technologies. The result is two data matrices that have the exact same set of individuals as rows, but different features (...
Nuclear Hoagie's user avatar
1 vote
0 answers
51 views

Finding the nearest point to a given set of points [closed]

I am trying to build an auto-suggestion logic which looks at currently selected items and recommends a list of items to select the next item from. I can formulate the problem thus: Given a set of ...
curious_explorer's user avatar
0 votes
2 answers
208 views

How to measure the similarity of frequency of categorical data

Sorry if this is a really simple question but I'm new to this and wondered if there's an easy way to do what I'm picturing. Imagine I've got a bunch of people and I'm asking them what they've eaten in ...
travelsandbooks's user avatar
0 votes
0 answers
58 views

Compare similarity or difference of two distributions by the ratio of moments

I'm looking for a measure that measure the similarity of two distributions in the following forms: $$ S = \frac{a \mu_1 + b \sigma_1}{a \mu_2 + b \sigma_2}. $$ The above formula I proposed is not ...
Wynn's user avatar
  • 1
1 vote
1 answer
90 views

Similarity measure for two discrete distributions (a group of proportions)

I am not really sure how to describe it with the correct mathematical terms. If there are any questions, I will try to explain further. I have several instances of a group of proportions. One could ...
Georg Schneeberger's user avatar
1 vote
0 answers
24 views

Dimensionality reduction that preserves non-trivial similarity

I have a set of vectors $x_i \in\mathbb{R}^m$ and a similarity function f that quantifies how similar $x_i,x_j$ are. Unfortunately, calculating f takes a lot of time. I want to use some neural ...
username_94's user avatar
2 votes
0 answers
37 views

Comparing the similarity of ranked word lists with a reference standard

Imagine a scientific organisation wants to communicate the likelihood of particular events occurring. For consistency, the organisation decides to always draw from the same set of words, which it ...
MoonResearch's user avatar
1 vote
0 answers
85 views

R adjusted cosine similarity [closed]

I would like to find an effective way to get adjusted cosine similarity for a large matrix (10k rows). Apply seems also slow... Could you please suggest a method? Maybe based on the approaches ...
Erin's user avatar
  • 11
4 votes
0 answers
47 views

Calculating similarity/resemblance of time series data? (soil moisture)

I have measured a variable (soil moisture) for multiple samples over time. Soil moisture increases steeply at several points in time (when it rains) and then decreases steadily until the next rain ...
empetrum's user avatar
1 vote
1 answer
38 views

Statistical argument for this cluster measure

This quick clustering score discussion presents the following single cluster scoring functions: $$ c_i = 1-\sqrt{\frac{\sum_{j}^{N}\left(1-\phi(i,j) \right)^2}{N-1}} $$ and $$ C = 1-\sqrt{\frac{\sum_{...
jman's user avatar
  • 123
0 votes
1 answer
177 views

How to compare two ranked lists of genes with ranking values?

I have 80 independent sets of genes with 6000 genes in each. Gene expression values are ranked and scaled from 0 to 1. So, a set looks like this: Gene1 1 Gene2 0.98 Gene3 0.85 ... Gene5998 0.002 ...
Yulia Kentieva's user avatar
4 votes
1 answer
208 views

Why do we not have a true upper limit for dissimilarity measure?

The similarity measure between two attributes always falls in the range of $[0,1]$, why is this not true in the case of dissimilarity where the value falls in the range of $[0, \infty)$? Can't the ...
Ashmin Jayson's user avatar
2 votes
0 answers
352 views

What does the PCA similarity factor tell us?

I found the following formula for comparing time series $$ S_{\mathrm{PCA}}^{\lambda}=\frac{\sum \limits_{i=1}^{k} \sum \limits_{j=1}^{k}\left(\lambda_{i}^{(1)} \lambda_{j}^{(2)}\right) \cos ^{2} \...
Unistack's user avatar

1
2 3 4 5
11