Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

Filter by
Sorted by
Tagged with
0
votes
1answer
34 views

What is the method should I use to calculate similarity in a data set with outliers that must be included?

Information: So I have a data set with 18 vectors with 167 components, each of with has a value with a range of $[-2, 2]$. I am trying to calculate the similarity between one arbitrary vector in ...
0
votes
1answer
35 views

Dynamic Time Warping vs Cross Correlation

Consider the following data frames: > df1 value 1 6 2 2 3 3 4 1 > df2 value 1 3 2 8 3 4 4 5 I want to find the ...
0
votes
0answers
36 views

How to measure consistency/similarity between 2 or more sets?

Say I have 2 sets of things that can't be ranked or anything. Each object in a set is unique but that object may or may not be in both sets. How to measure the consistency between the two sets? By ...
0
votes
1answer
40 views

multiple measurements for classification; which one to choose?

There are five different classes (labels) where the new points have to be classified at. For each new point, five different measurements are conducted, resulting into similar although different values ...
0
votes
0answers
16 views

Compute simliarity between two sets of numbers with each set summing up to 100%?

I have product shares for different users. Need to find similarity between users based on these product shares. For Example, User 1 Prod A: 44% Prod B: 33% Prod C: 23% User 2 Prod A: 76% Prod B: 24%...
0
votes
1answer
43 views

How to compare two bayesian linear models?

Let say, we have two bayesian linear models learned from some data (not necessarily same but from the same data distribution), how can we compare or what is the notion of similarity between the two ...
0
votes
1answer
19 views

Interpretation of clustering on similarity matrix

Many papers suggest doing clustering not on the (n x p)raw data but on the n x n or p x p matrices computed according to determined similarity measures (eg, correlation, cosine, point mutual ...
0
votes
0answers
22 views

Can Bhattacharyya coefficient (or distance) be used as an additive measure to compute a metric for performance?

As far as I understand, Bhattacharyya's measure(s) can be used to see similarity between two empirical distributions. Other ways to do so are nicely explained here: Similarity measure between multiple ...
0
votes
1answer
34 views

Is Hierarchical clustering a special case of knn(specific n=1)?

I'am working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
0
votes
0answers
9 views

Vector representation of subsets of data in python

I have 141575 data points that can be divided into 153 blocks. There are around 70 features in my dataset. I want to measure similarity among the blocks and rule out outliers during training my model. ...
2
votes
2answers
99 views

How different will that be between the R-squared of linear regression y~x and square of cor(x,y)

Generally, both of them can represent the linear relationship between x and y scale to [0,1]. Are they 99% very similar?
0
votes
0answers
19 views

Set Similarity Measure that accounts for size

Suppose I have two sets, A and B where 10|A| = |B| and A is a subset of B. For the case where |A| = 1 the Jaccard similarity of the sets will be 0.1. For the case where |A| = 100 the Jaccard ...
0
votes
0answers
10 views

Practical handling of Dynamic Time Warping for time-series with unequal sampling frequency (irregular time series)

Hi all and thank you for taking time to read this. When I read the general literature about dynamical time warping (and the vignette for the $\texttt{R}$ package $\texttt{dtw}$), the time series seem ...
0
votes
0answers
16 views

How to Measure/Quantify Similarity between > 2 Sets of Dummies Indicators

I have multiple sets, each with a binary indicator for whether an element is part of the set. ...
0
votes
2answers
37 views

Clustering high dimensional data

I was going through this wiki page on clustering in high dimensions and I don't understand the following statement there. Can someone explain to me what this means? The concept of distance becomes ...
0
votes
1answer
36 views

Similarity between n >> 2 binary vectors

I have $n$ binary vectors $x_i, ..., x_n$ of length $d$. I wish to compute a similarity measure between all $n$ of them. My initial thought was to use the Jaccard Index, since each binary element $x_{...
0
votes
0answers
28 views

Similarity of form of two curves

Currently I'm working on a project on finding out the similarity of two movies based on the sentiment analysis I did on their subtitles. I measured the sentiment of each individual sentence and paired ...
0
votes
1answer
55 views

Comparison of two normal distribution

I have two normally distributed samples. I want to know how close or similar it is. I tried few methods to find the similarity, like z-score and bhattacharyya distance. Bhattacharyya distance didn't ...
1
vote
1answer
134 views

Calculate Earth Mover's Distance for two grayscale images

I am trying to calculate EMD (a.k.a. Wasserstein Distance) for these two grayscale (299x299) images/heatmaps: Right now, I am calculating the histogram/distribution of both images. The histograms ...
2
votes
1answer
62 views

Statistically compare similarity between images

I have two images/heatmaps (2d matrix) of identical size. I need to statistically compare the similarity between the two. With 'similarity', I mean that high and low values of one image appear in ...
0
votes
1answer
16 views

similarity function for keywords that often occur together

I'm working on a project where I have to develop a machine learning solution. I should mention that I'm not very experienced in ML yet and that I'm not even sure if the term "similarity function" is ...
0
votes
0answers
15 views

Dimensionality of similarity matrix

Below is a screen shot of a paper. The authors take a data-set $E\in R^{nxm}$. Here $n$ is the number of observations/samples/patients and $m$ is the number of genes/features. Preprocessing ...
0
votes
1answer
20 views

What distance function to use for matching observations on a mix of ordinal, nominal and numeric variables?

I'm trying to match potential respondents to a random sample from the target population, and not sure whether any of the usual methods available out-of-the-box in e.g. R are right for the job. ...
0
votes
0answers
10 views

Term x term matrix for text clustering. What to do with subterms of n-grams

I am doing topic discovery on a large corpus. Reading here and there I found some papers saying that in case of big, sparse document x term matrices is better to create first a term x term similarity ...
0
votes
0answers
26 views

Clustering/Similarity between drivers

I have a dataset that contains initial and ending points of car trips: ...
1
vote
0answers
28 views

Is it possible to evaluate similarity (as in Kolmogorov-Smirnov test) between two sample vectors using copulas?

Consider the Two-sample Kolmogorov-Smirnov test: given 2 vectors of samples $X$ and $Y$ it is possible to test whether they are drawn from the same continuous distribution by calling (in R, for ...
2
votes
1answer
165 views

What GLM family and link function to use with similarity index as response variable?

I need to model an unusual response variable (at least to the best of my knowledge) that is: similarity (estimated by Morisita-Horn index) in species composition between pairs of sites. My response ...
0
votes
1answer
11 views

String similarity measure for pitch contour similarity (Parson code)

I have a dataset with short melodies hummed by experimental subjects. Each melody consists of a variable number of notes (10 to 20 notes) and is coded by the pitch levels of the notes (automatically ...
0
votes
0answers
14 views

Using the appropriate method for measuring the similarity of 1-D discrete time series data

I have a data of a 1-D time series data (more than 4000 points), they have fixed spacing, sample at fixed frequency. My task is to 'extract' the part in this data that match/look similar to my input ...
1
vote
0answers
24 views

R daisy - Gower distance, different values with different number of observations [duplicate]

I have a mixed data set where I want to compute distances between observations with Gower in R (daisy function). When I compute distances between different number of observations, the distances seem ...
1
vote
0answers
12 views

What can I use to compute a similarity (or diversity) index for a sample with “multidimensional” attributes?

Current problem: We have a batch of $n$ items for which we capture their details with $m$ attributes. It could look something like this: The goal is to compute an "index" that says how "similar" this ...
0
votes
0answers
6 views

How to comparing the similarity between two predictions with 2 fixed+1 alternative outcomes?

I want to compare the similarity of two diseases to Parkinson's disease (PD) with the same predictors. My first instinct is to run a multinomial regression with two fixed ends (healthy control and ...
0
votes
0answers
29 views

Similarity measure before and after dimensionality reduction or clustering

I have a dataset with 500 000 samples, each sample contains 30 features. The values of the features are in the range 0.0 to 1.0. ...
2
votes
1answer
56 views

Distance metric with characteristics of cosine and Manhattan

I'm working on a project where I want to find similarities between groups of events. So far I have expressed groups of events as vectors of event counts and computing similarities between them. I'm ...
0
votes
0answers
18 views

Mahalanobis: Covariance or correlation? [duplicate]

Is the Mahalanobis distance using correlations or covariances (among two vectors) to determine the similarity? I know that both are quite the same and differ only by a division of a standard deviation ...
1
vote
1answer
21 views

Clustering algorithm for similarity measure on quantitative values

I have a big set of subsets of 2 or more documents (maximum 10). The documents from each subset should describe the same product. One document contains pairs attribute - value: ...
0
votes
0answers
11 views

Correlation-like characteristic for ordinal feature with only three values

I have tabular data with two variables: one of them is regular continuous (float) variable and the second one is ordinal with three values only. I want to measure their dependency. (If it can help, I ...
0
votes
1answer
60 views

Calculating similarity between two lists: high cosine similarity, but high RMSE

I want to see how similar two datasets are, as a way to justify that they can be used in similar contexts. In practice one dataset contains manually calculated data, and the other automatically ...
0
votes
1answer
112 views

Function of Matrix Profile Index and Joins

I would like to find similar motifs in a time-series using Matrix Profile, but by selecting the sub-sequence manually, instead of doing dimension reduction. Matrix Profile From what I understand, ...
0
votes
0answers
28 views

Different backgrounds, different similarities with ecospat (R language)

I beg all of you your excuses, because the stupid question. I am exploring niche conservatism in five species of Ericaceae. I created two backgrounds sets in order to explore niche similarity (using R ...
1
vote
1answer
277 views

How to use Matrix Profile for dimension reduction and clustering

Matrix Profile (MP) has been used for clustering time-series segments. In the slideshow tutorial featured on the MP website, they use a figure to demonstrate projecting segment similarity onto an M-...
0
votes
0answers
16 views

Is there a method that's better than clustermaps for clustering samples which have a n of variables but only two possible outcomes?

For each variable there are two possible outcomes 1 or 0 (True or False): It shows the appearance of particular species on different localities. Mine goal is to cluster/classify similar localities ...
0
votes
1answer
89 views

Text Similarity - Cosine - Control. Suggestion to another / better method?

I would like to ask you, if anybody could check my code, because it was behaving weird - not working, giving me errors to suddenly working without changing anything - the code will be at the bottom. ...
0
votes
1answer
33 views

Similarity index between two texts Ask Question

I'm trying to compare two vectors in a small NLP project using Python. Code doesn't make any difference since I'm using scikit-learn, but my doubts are about my calculations. I have a query vector ...
0
votes
1answer
41 views

Similarity between 2 profiles (observations). Is it possible to generate a % similarity?

I have multiple profiles for 10 different people. Each person has been measured for 5 different continuous variables of different magnitudes. So my dataframe is 10x5 where each row represents a person ...
0
votes
0answers
22 views

Somers' D deteriorating for more granular rankings

Background I have 30 observations and I want to compare how much experts and model agree with each other. For Model and Expert I have slots which are $\in [1,4]$ and ranking which are $\in [1,30]$. ...
1
vote
0answers
52 views

Similarities / suggestions of multi-attribute entities

Dataset: I have 200 000 entites, these have zero or more of 1000 attributes (commonly 1-20 of them) The attibute relation is weighted 1-3 Its stored in a relational database modeles as Entity, ...
2
votes
0answers
81 views

Similarity between Train and Test data sets

I have multiclass classification dataset and I am using Deep nets for the classification task. To explain the problem, let's assume that I have 5 classes to classify. No matter what I try, be it ...
-2
votes
1answer
54 views

Find similar images in a dataset without labels [closed]

I have a set of grayscale images, some of them are transformed of the other images. For example in 10 images, image 2 is the same as image 8 but rotated, and image 4 is the same as image 7 but ...
0
votes
1answer
53 views

Clustering visited vertex in a graph

I would like to know can anyone introduce me an algorithm to cluster multiple sequence of visited vertices in a graph. For example assume your graph has 6 nodes. following might be 3 different ...