Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Compare differences between 2 datasets with Jensen-Shannon divergence

I have two medical imaging datasets: Brain MRI from institution A Brain MRI from institution B Because of different acquisition parameters the intensity distribution in those datasets could be ...
0
votes
0answers
13 views

How to find the similarity?

I have a problem statement but I am not sure how to approach it accurately. Let me explain the statement. Assume that I have a website that many shops are selling books. Some books are very demanded,...
2
votes
0answers
34 views

How to measure similarity between tag distributions?

I have a table of movie tag assignments where each row is (userId, tagId, movieId). I'm using test data set from Movielens with a million rows. I want a function that given two films returns a ...
0
votes
0answers
14 views

Similarity index between 2 unevenly sampled time series

Say I have 2 time series $s_1$ and $s_2$ with independent variable $x_1$, and dependent value $y$. These 2 series are not evenly sampled across $x_1$, or even sampled at the same rate. Now, I have ...
0
votes
0answers
27 views

Statistically, how to find a subset of values which seems to have same magnitude?

Data Set Description I have a data set where students' spent time on study is available. In this table, student 4 to student 10 seem to spend similar amount of time on study. There is very little ...
1
vote
0answers
11 views

Anconva or Anova for multifactor analysis

I have non-normal plankton community data, where rows are sites and columns are species (dependent) and a few factors. I want to check the influence of season on community (dis)similarities but am ...
0
votes
0answers
6 views

Is there a way to determine the most similar variables from a number of different time-series?

I have a time-series data frame with 27 different variables. In reality, they are all different instruments recording the same measurand every 5 minutes. I want to know if there is a way of ...
1
vote
0answers
23 views

Check which parameter causes less dispersion in a sample set - When to use CV (Coefficient of Variation) or CD (Coefficient of Dispersion) etc

The overall goal is to extract/engineer features from approximately 100 segments (various lengths but always more than 80 data points) that are as similar as possible to each other and have a very low ...
1
vote
0answers
13 views

Similarity between clusters/groups?

I have a dataset consisting of multiple groups in a high dimensional space. An example is shown below: What would be the best way to calculate similarities between groups. Say how similar is group A ...
1
vote
1answer
23 views

Similar histograms after removing mean and dividing with std

I have four time-series, of a physical quantity (heat flux). When I remove the mean value from my data points Xn = X - mean(X) and divide them with the standard deviation Xn' = Xn/std(Xn) the ...
0
votes
1answer
43 views

Using similarity matrix as a correlation input

I am using an approach to decompose sigmoidal signals in R. Briefly, signals are decomposed into a subset of components and then a custom value of similarity is computed among samples defined by 2 ...
0
votes
0answers
56 views

How to avoid non-independence of data points when measuring alignment?

I am trying to compare groups on alignment. A group has high alignment to the extent that members of the group are similar to one another. A group has low alignment to the extent that members of the ...
0
votes
0answers
15 views

How to group products with similar attributes?

Customers ask me for a quote for a a product. I might have quoted a nearly identical product for a different customer already albeit with slightly different attributes. Only in rare cases they're ...
1
vote
0answers
33 views

how does VAT algorithm works

I want to know how the VAT algorithm for cluster tendency works in detail. As you can see in the picture below R is the dissimilarity matrix and R-tilda is the ordered dissimilarity matrix. What is ...
1
vote
0answers
19 views

Kullback-Leibler Divergence vs Normalized Cross Correlation

I have couple of time series data that I want to cluster. As I was looking for ways to calculate similarity for time series data, I came across couple of different similarity methods. What I am ...
0
votes
0answers
13 views

Compare similarity between two different clusters having few similar fields?

I have two datasets, one is mentor dataset another is jobseeker dataset. My ultimate goal is to recommend a jobseeker to mentor based on skills and location (skills jobseeker has but need mentorship ...
0
votes
0answers
20 views

Does DTW return smaller distance measure than Euclidean Distance?

QUESTION 1: When computing the distance between two time series, shouldn't the DTW distance measure return a smaller distance than the Euclidean distance (assuming DTW internally uses the Euclidean ...
0
votes
0answers
19 views

Finding weights to be used by an Euclidean distance function for vectors with weighted components in a multi-dimensional space, using only sample data

I have a multi-dimensional space with $d$ dimensions wherein $i$ vectors ($v_1 ... v_i$) with $d$ components live. I want to find a function $s(v_a, v_b)$ which takes in two of these vectors and ...
1
vote
0answers
14 views

the OKCupid formula (and other ways of estimating similarity with limited data)

Say I have a test, that receives a string and returns either Belongs or Does Not Belong (to a set) I know the true answers (if each sample belongs to the set or not), and I want to evaluate the ...
0
votes
0answers
10 views

Finding the relative 'importance' of different vector components when defining distance of two vectors in a space [duplicate]

I have a multi-dimensional space with $d$ dimensions wherein $i$ vectors ($v_1 ... v_i$) with $d$ components live. I want to find a function $s(v_a, v_b)$ which takes in two of these vectors and ...
0
votes
0answers
27 views

Is there a way calculate a gradient of dissimilarity across multiple sites using presence/absence data?

I’ve been trying to come up with a metric for species (dis)similarity for fish in a series based on presence or absence data that gives a value of 1 or 0 for each lake. From my research, it seems like ...
0
votes
0answers
11 views

Finding distance/similarity function between two vectors in a multi-dimensional space, with weighted components/dimensions [duplicate]

I have a multi-dimensional space with $d$ dimensions wherein $i$ vectors ($v_1 ... v_i$) with $d$ components live. I want to find a function $s(v_a, v_b)$ which takes in two of these vectors and ...
1
vote
1answer
31 views

Metric for distance of sinewaves

I have time-series data consisting of the sum of 2 sinewaves and my goal is to predict their frequencies and their amplitudes. I would like to know what are the best distance metrics/loss functions I ...
1
vote
0answers
21 views

Similarity measure in classification: when do you need it to be a metric?

I am currently considering using dynamic time warping (DTW) for outlier detection in time series data. I am fairly new to the technique and have been reading: That I shoud be cautious using DTW since ...
0
votes
0answers
17 views

Similarity measure for two lists of scores

I have a finite set of items, a score for each item, and a series of orderings: ...
0
votes
0answers
13 views

I don't understand the task of finding the correlation in WordSimilarity-353

My issue is the following, I have created word embeddings using WordNet, and to test my embeddings to see how they stack up against the word similarities in WordSimilarity-353 I'm running a script ...
1
vote
1answer
55 views

Rao-Stirling diversity index

I need help to calculate the Rao-Stirling diversity index. I tried it several times but cannot achieve equal results to R packages yet (e.g. diverse). I used the Rao-Stirling diversity index as ...
1
vote
2answers
126 views

Understanding the R stats mahalanobis() function's Output

An acquaintance recommended I use the Mahalanobis distance on my data instead of Euclidean, Manhattan, etc. I tried using the mahalanobis() function in the R stats package on a data matrix with N ...
1
vote
0answers
25 views

Dynamic Time Warping and non-stationarity of time series

Dynamic Time Warping (DTW) computes the optimal alignment between points of two time series. If we consider two non-stationary time series (e.g. the first proportional to the second), is the DTW ...
0
votes
0answers
4 views

RBF-Kernel: Handling missing values

I want to compute the RBF-Kernel for a dataset which contains missing values: ...
1
vote
0answers
40 views

Distance measure for two probability distribution of unequal sample size

Context: I have 100 stores and these stores are divided into 10 business markets. I want to select 3 markets where each market is a good representation of the 100 stores i.e. the population. There ...
2
votes
1answer
215 views

Finding similarity between two datasets [closed]

I have two datasets. One is actual percentage of white population in counties in an american state and the other is the simulated percentage of white population in counties in an american state. ...
1
vote
0answers
17 views

Probability that output is result of the same process?

I've developed a simple Monte-Carlo simulation. Output of this simulation is a histogram. This histogram is possibly a log-normal distribution, but I don't want to assume that. But I do know that the ...
0
votes
1answer
39 views

Training classifier on randomly generated negative samples

I have $M$ (~dozen million) feature vectors. There are $F$ (~several dozen thousand) binary features, but in each vector, only $H$ (~several hundred) of them would be 1, the rest are 0. Now, for a ...
0
votes
0answers
17 views

Understanding formula for comparing players based on certain metrics

I'm looking at this tool: https://www.capfriendly.com/comparables/ and trying to understand the formula behind its rankings. I don't really need the exact formula, I really just need a way to compare ...
0
votes
1answer
62 views

Cosine similarity vs quadratic objective function

The aim is to calculate the similarity between two foods given the nutritional content of each. After some reading, it seems the most popular measure for this sort of problem is the cosine similarity ...
2
votes
1answer
153 views

FindSimilar items in a complex dataset

I'm a Machine Learning newbie, but I want to learn more about this interesting topic using a practical example, on which I would appreciate any theoretical and practical help: I have a database of "...
0
votes
0answers
43 views

Weigh Kullback Leibler Divergence with P entropy?

I'm wondering if it makes sense to weight Kullback-Leibler Divergences to highlight divergences on highly distinguishing features. I am however not very well versed in Information Theory, and would ...
1
vote
0answers
36 views

Name of a similarity test for Ordinal data to see how similar a set of participant answers are to expert answers

I asked 12 participants in a small experiment to order 5 distorted webpages from the least to most trustworthy (1 - 5). I want to know the best way to test how similar participants answers are to a ...
1
vote
1answer
108 views

How do I statistically compare the readings from two different sensors? [duplicate]

I have temperature and humidity readings from two different sensors (located in the same geographical area). The data is logged every hour and I have data for more than a year, logged this way 2/1/...
0
votes
1answer
66 views

How to find similar users in a social network

I have a set of users from a social network. These users are represented by large sparse vectors. Let's say that a small subset of those users bought a ticket for a particular movie. How could I find ...
0
votes
1answer
40 views

What is the method should I use to calculate similarity in a data set with outliers that must be included?

Information: So I have a data set with 18 vectors with 167 components, each of with has a value with a range of $[-2, 2]$. I am trying to calculate the similarity between one arbitrary vector in ...
1
vote
1answer
441 views

Dynamic Time Warping vs Cross Correlation

Consider the following data frames: > df1 value 1 6 2 2 3 3 4 1 > df2 value 1 3 2 8 3 4 4 5 I want to find the ...
0
votes
1answer
71 views

multiple measurements for classification; which one to choose?

There are five different classes (labels) where the new points have to be classified at. For each new point, five different measurements are conducted, resulting into similar although different values ...
0
votes
0answers
16 views

Compute simliarity between two sets of numbers with each set summing up to 100%?

I have product shares for different users. Need to find similarity between users based on these product shares. For Example, User 1 Prod A: 44% Prod B: 33% Prod C: 23% User 2 Prod A: 76% Prod B: 24%...
0
votes
1answer
118 views

How to compare two bayesian linear models?

Let say, we have two bayesian linear models learned from some data (not necessarily same but from the same data distribution), how can we compare or what is the notion of similarity between the two ...
0
votes
1answer
361 views

Interpretation of clustering on similarity matrix

Many papers suggest doing clustering not on the (n x p)raw data but on the n x n or p x p matrices computed according to determined similarity measures (eg, correlation, cosine, point mutual ...
0
votes
0answers
89 views

Can Bhattacharyya coefficient (or distance) be used as an additive measure to compute a metric for performance?

As far as I understand, Bhattacharyya's measure(s) can be used to see similarity between two empirical distributions. Other ways to do so are nicely explained here: Similarity measure between multiple ...
0
votes
1answer
314 views

Is Hierarchical clustering a special case of knn(specific n=1)?

I'm working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
0
votes
0answers
11 views

Vector representation of subsets of data in python

I have 141575 data points that can be divided into 153 blocks. There are around 70 features in my dataset. I want to measure similarity among the blocks and rule out outliers during training my model. ...

1
2 3 4 5
8