Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

Filter by
Sorted by
Tagged with
1 vote
0 answers
24 views

How to statistically compare two numerical maps? [closed]

I have spatial temperature data on a relative scale from 1 to 0 on several days. The data sets have the same resolution, origin, and extent. How can I statistically compare them? How to identify ...
user avatar
  • 11
0 votes
1 answer
29 views

Combining similarity scores from 2 measures into one score [closed]

Assume I have items: a, b, and c. And I calculated the similarities between each pair using two different measures, and I got the following similarity score metrics:...
user avatar
1 vote
1 answer
10 views

High Performance Classification or Similarity Algorithim for Mixed Data Types?

I have a database holding 10-ish features that describe different breeds of dogs. They are mostly categorical features, but some provide ranges for values. Here's a demo representation of the database,...
user avatar
0 votes
0 answers
13 views

Possible ways to feed two variables containing text data into an ML model in an NLP problem

In a natural language processing (NLP) problem, we have a couple of variables, say A and B. A denotes a phrase (1-2 words) and B denotes another phrase (>3 words). There is one target variable that ...
user avatar
0 votes
0 answers
13 views

Measure for similarity of two distribution ECDF with sensitivity to the tails

Currently, I am looking for a measure to quantify the overall dissimilarity or similarity of two sample distributions (possibly of different size). I would like to compare observed data with model ...
user avatar
  • 11
0 votes
0 answers
19 views

Comparison of normality + similarity distribution of time series datas

I have a training time series data , whose last 20% is kept as validation data. I want to check whether the distribution of training and validation features are similiar and normal, so that we can say ...
user avatar
  • 1
0 votes
0 answers
15 views

What kind of information can be extracted from a similarity matrix?

Say, you have an N×N similarity matrix. What kind of information can you extract from a similarity matrix and what is its intuitive meaning? For instance, one could compute trace (meaningless), ...
user avatar
0 votes
0 answers
6 views

What kind of information can be extracted from a similarity matrix?

Say, you have an $N\times N$ similarity matrix. What kind of information can you extract from a similarity matrix and what is its intuitive meaning? For instance, one could compute trace (meaningless),...
user avatar
7 votes
2 answers
2k views

Kolmogorov-Smirnov Test in Python weird result and interpretation

I have difficulties to understand how the Kolmogorov-Smirnov Test works. If I want to know if my samples are from a specific distribution (for example from the weibull distribution) I can compare my ...
user avatar
  • 73
1 vote
1 answer
27 views

similarity measures for comparing multiple distributions

I have a set of random variables $(X,C) \in \mathcal{X} \times \mathcal{C}$, where $X$ can be continuous or discrete and of multiple dimensions, and $C$ is discrete. Which are appropriate measures for ...
user avatar
  • 13
0 votes
0 answers
23 views

Similarity measures for Probability Mass Functions

I am trying to predict whether two sets of papers are written by the same author by looking at the distribution of papers over the years (number of papers published in a given year). Suppose we have ...
user avatar
0 votes
0 answers
45 views

Scalar similarity measure between two vectors including both angle and magnitude

I have different models, predicting a vector $\boldsymbol{v}\in\mathbb{R}^3$. Now I would like to compare the performance of these models against a baseline vector $\boldsymbol{b}\in\mathbb{R}^3$, for ...
user avatar
  • 251
0 votes
0 answers
27 views

How to measure whether a set of vectors are more similar to each other than a set of randomly selected vectors would be?

I have ~15 000 word embedding vectors of length 256, that can be categorized into several groups (sizes 2 to 1500) via heuristic considerations. My aim is to measure whether the embedding vectors ...
user avatar
1 vote
0 answers
19 views

Similarity between datasets A and B where B is a subset of A

I have two datasets A and B, and for each entry in both datasets I have a mixture of ordered and unordered categorical variables such as gender, age (integral value) and date. It is believed that ...
user avatar
1 vote
1 answer
39 views

fast distance metric between a new data entry and available clusters of data

Assume we divide a large data set D into m different partitions of data in a distributed learning case, using ...
user avatar
  • 125
0 votes
1 answer
24 views

Variance used in t-SNE

I was going through the t-SNE, but I am bit confused. While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point ...
user avatar
0 votes
0 answers
24 views

Quantify similarity between time-series with uneven sampling

I would like to find a way to numerically represent the similarity between three time series, with differing sampling rates. In the past I have used other methods in R, including dynamic time warping, ...
user avatar
  • 113
0 votes
1 answer
22 views

Measuring within-group similarity for ranked list responses

Individuals are completing a task in which they rank 15 items (1-15 according to importance). These individuals will later be working in groups of 4, so I am interested in measuring the similarity of ...
user avatar
  • 1
0 votes
0 answers
19 views

Same value in independent variables for one section of the respondents

I am planning to use 6 indexes (community index) as independent variables for 12 communities. And planning to run regression where the dependent variable will be collected from the individual ...
user avatar
0 votes
0 answers
32 views

Comparing fit of time series to true process

I know that this question have been asked many times in this forum, but I am having troubles in understanding the correct approach to my aim. I have several time series representing a growth process ...
user avatar
1 vote
0 answers
33 views

Quantify similarity of two time series

I am a biologist that is not proficient on statistics or using R, so bear with me. I have data for two factors that increase or decrease over time. I have replicate measurements of both factors at the ...
user avatar
0 votes
0 answers
11 views

Pearson correlation coefficient as a measure of similarity between vectors? [duplicate]

I'm trying to understand, in layman's terms, what it is that pearson correlation coefficient is actually measuring and how it's used as a measure of similarity between two vectors. I don't have much ...
user avatar
  • 101
0 votes
0 answers
15 views

Evaluating similarity of sequences based on whether the sequences could have been drawn from the same probability distribution

I have multiple sequences with elements drawn from a specific set (e.g. A, B, C, ...). The length of the different sequences is not necessarily equal. Typical lengths are on the order of 100 to 400 ...
user avatar
0 votes
0 answers
21 views

How Can I Reduce Similarity Analysis of Multiple Time-Series Vectors into a Single Value?

I have ~15 independently-sourced vectors with about 1600 samples in each. They are basically continuous, ~1 Hz, from t=0 to 22 minutes. The nature of the dataset is such that the signals are ...
user avatar
0 votes
0 answers
32 views

kernel PCA similarity matrix analogy

The standard explanation to linear PCA begins with the covariance matrix. That is, for a dataset $D$ of dimension $N \times d$, the covariance matrix is given as $\sum = \frac{D^{T}D}{N}$ where the ...
user avatar
  • 207
0 votes
0 answers
30 views

How do I identify if, out of three groups, two are more similar to each other than the other, with statistical support, in R

I have seen similar questions but I believe there are some differences in what I wish to do. I have three groups, A,B, and C. I have two variables namely matings, feather reflectance for all the ...
user avatar
  • 3
0 votes
0 answers
8 views

Event Syncronization among binary variables

Let's suppose I want to study if two event happen approximately at the same time. This is a potential dataframe, I coded as 0 no-event and 1 event ...
user avatar
0 votes
0 answers
13 views

Why very different cosine similarity scores when: mean(cos_sim(A, B), (A, C)) than when comparing to concatenated: cos_sim(A, concat(B, C))

I am trying to compare rhetorical similarity of politicians to the rest of their party in 2010. Say party 1 consists of politician A, B and C. Each politician has made one speech in 2010. If I want to ...
user avatar
  • 1
1 vote
0 answers
27 views

Metric to assess similarity of distributions

I am working on clinical synthetic data and I would like to learn more about metrics to compare synthetic vs measurements distributions. As there are methods to generate synthetic distributions with ...
user avatar
1 vote
2 answers
58 views

What would be a reason to use the Root Mean Square Error (RMSE) to combine data?

In this machine learning tutorial by Google they use the Root Mean Square to create a similarity measure between two shoes. They first calculate the difference between the size of two shoes and then ...
user avatar
0 votes
1 answer
45 views

Clustering methods for regional temporal clusters

I have observations of individuals over time, where they either experience -1, +1 or mostly 0, like so. ...
user avatar
0 votes
0 answers
17 views

Kolmogorov-smirnov test to compare how similar different pairs of samples are

Say I have a list of samples in pairs: [(A, B), (C,D), (E, F)], where A,B,C,D,E,F are different samples and can be of different sizes. Now if I want to compare which of these pairs are statistically ...
user avatar
  • 101
0 votes
0 answers
11 views

What are general guidelines to construct a unique measure of similarity for your data?

I'm taking an introductory course in data science. As part of an assignment I'm given a dataset where I must conceive a new measure of similarity (no cosine similarity, or euclidean distances allowed, ...
user avatar
2 votes
1 answer
82 views

Obtain a between-class similarity. And is the way to do it through PCA valid?

Context: I have a dataset containing instances labeled into different classes, and for all the classes, I have the same set of features. My research question is to identify classes that are more ...
user avatar
  • 129
0 votes
2 answers
46 views

Find similarity between two datasets [closed]

In data science or statistics, what techniques are there to find the similarity between two datasets? I.e. let's say I have 2 sets of data, a) max temperatures for the month of February and b) max ...
user avatar
  • 109
0 votes
0 answers
58 views

How to measure similarity between two sets?

I want to measure the similarity between sets of images. Every set contains unique image names. The biggest challenge for me is that the sets are not equally sized, and not all images of setA are in ...
user avatar
5 votes
3 answers
347 views

Measure of similarity between two distributions - with variable starting point (wrap-around)

I would like to compute the (dis)similarity between two distributions. The first represents tones sung by an experimental subject during a recording. There are different numbers of tones, each with ...
user avatar
  • 1,548
0 votes
0 answers
23 views

Combining two gram matrices with different dimensions for machine learning

I want to classify pairs of items $(x_1, x_2) $ with $ x_1 \in X_1$, $x_2 \in X_2$. I know the similarities (gram matrices) of the items in $X_1$ and $X_2$: $K_1(x_{1,i}, x_{1,j}) = \phi_1(x_{1,i})^T\...
user avatar
  • 423
0 votes
0 answers
26 views

Can you model a correlations as a dependent variable?

I am working on a dataset where each the data for each individual can be seen as a vector of numbers between zero and one. To be precise, this is the methylation level at each cytosine in the genome. ...
user avatar
  • 165
0 votes
0 answers
27 views

How to match a curve/image to part of another curve/image?

I have a long curve and want to extract segments with specific feature, the feature is given by a template curve. I intend to use similarity measures like Fréchet distance or dynamic time warping to ...
user avatar
  • 101
2 votes
1 answer
46 views

efficient method for sentence similarity

I have a file with about 25000 rows. I want to find sentences that are near duplicates (sentences which have difference of just one or two words/symbols or ones which are just paraphrasing of ...
user avatar
0 votes
1 answer
18 views

Inter-participant similarity for multiple continuous values

I am curious to see how similarly my participants process a given experiment. (I do not think "agreement" is the right term here so I used similarity.) The values are continuous and have no (...
user avatar
0 votes
0 answers
38 views

How to test the similarity between two labelled datasets?

I have two different functions with I can draw labelled data $(x, y)$ from, where $x$ is multi-dimensional, and $y$ is a real number value. There exist a $y$ value for any possible $x$ in the input ...
user avatar
0 votes
0 answers
17 views

Measure for difference or similarity between two regions of the plane

Suppose I have a picture and let two people mark regions within it. How can I measure the (dis)similarity of the two marked regions? More mathematically, suppose I have a (filled) rectangle $A\subset \...
user avatar
  • 226
3 votes
1 answer
145 views

When to use dot-product as a similarity metric

I'm trying to understand which similarity measure should be used in which situations. Roughly speaking, when should someone use the dot product to assess similarity between vectors? Roughly speaking, ...
user avatar
  • 628
0 votes
0 answers
47 views

How should one normalize activations of batches before passing them through a similarity/distance function? (like CCA, CKA, OP)

I was reading these two paper: https://arxiv.org/abs/1905.00414 https://arxiv.org/abs/2108.01661 and they mention that before computing the similarity of two layers in a neural network ...
user avatar
0 votes
0 answers
35 views

Two sample similarity analysis for non-independent daily time series data

What two sample or other relevant statistical tests can be performed in R between two samples which are independent of each other, but the observations within each ...
user avatar
1 vote
0 answers
147 views

Cosine similarity with vectors of different lengths (time series)

I have time series of different lengths with different patterns. I want to find the similarity between them in such a way that it is defined from 0 to 1 and the measure takes into account the length (...
user avatar
  • 23
0 votes
0 answers
31 views

Quantitatively comparing two time series

Summary: I'm trying to quantitatively compare two data series by looking at (a) the difference between the lines and (b) whether they follow the same trends Details: I have multiple time-series data ...
user avatar
1 vote
0 answers
23 views

How to find which today's time series value is relatively more different?

I have 130 times series, each present daily average price of a stock till today, They have very very similar behaviour in terms of trends, peaks, minimums .... just the level are different (y value). ...
user avatar
  • 11

1
2 3 4 5
10