Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

0
votes
2answers
13 views

How different will that be between the R-squared of linear regression y~x and absolute value of cor(x,y)

Generally, both of them can represent the linear relationship between x and y scale to [0,1]. Are they 99% very similar?
0
votes
0answers
12 views

Set Similarity Measure that accounts for size

Suppose I have two sets, A and B where 10|A| = |B| and A is a subset of B. For the case where |A| = 1 the Jaccard similarity of the sets will be 0.1. For the case where |A| = 100 the Jaccard ...
0
votes
0answers
5 views

Practical handling of Dynamic Time Warping for time-series with unequal sampling frequency (irregular time series)

Hi all and thank you for taking time to read this. When I read the general literature about dynamical time warping (and the vignette for the $\texttt{R}$ package $\texttt{dtw}$), the time series seem ...
0
votes
0answers
14 views

How to Measure/Quantify Similarity between > 2 Sets of Dummies Indicators

I have multiple sets, each with a binary indicator for whether an element is part of the set. ...
0
votes
2answers
29 views

Clustering high dimensional data

I was going through this wiki page on clustering in high dimensions and I don't understand the following statement there. Can someone explain to me what this means? The concept of distance becomes ...
0
votes
1answer
21 views

Similarity between n >> 2 binary vectors

I have $n$ binary vectors $x_i, ..., x_n$ of length $d$. I wish to compute a similarity measure between all $n$ of them. My initial thought was to use the Jaccard Index, since each binary element $x_{...
0
votes
0answers
25 views

Similarity of form of two curves

Currently I'm working on a project on finding out the similarity of two movies based on the sentiment analysis I did on their subtitles. I measured the sentiment of each individual sentence and paired ...
0
votes
1answer
52 views

Comparison of two normal distribution

I have two normally distributed samples. I want to know how close or similar it is. I tried few methods to find the similarity, like z-score and bhattacharyya distance. Bhattacharyya distance didn't ...
1
vote
1answer
72 views

Calculate Earth Mover's Distance for two grayscale images

I am trying to calculate EMD (a.k.a. Wasserstein Distance) for these two grayscale (299x299) images/heatmaps: Right now, I am calculating the histogram/distribution of both images. The histograms ...
2
votes
1answer
50 views

Statistically compare similarity between images

I have two images/heatmaps (2d matrix) of identical size. I need to statistically compare the similarity between the two. With 'similarity', I mean that high and low values of one image appear in ...
0
votes
1answer
14 views

similarity function for keywords that often occur together

I'm working on a project where I have to develop a machine learning solution. I should mention that I'm not very experienced in ML yet and that I'm not even sure if the term "similarity function" is ...
0
votes
0answers
11 views

Dimensionality of similarity matrix

Below is a screen shot of a paper. The authors take a data-set $E\in R^{nxm}$. Here $n$ is the number of observations/samples/patients and $m$ is the number of genes/features. Preprocessing ...
0
votes
1answer
15 views

What distance function to use for matching observations on a mix of ordinal, nominal and numeric variables?

I'm trying to match potential respondents to a random sample from the target population, and not sure whether any of the usual methods available out-of-the-box in e.g. R are right for the job. ...
0
votes
0answers
10 views

Term x term matrix for text clustering. What to do with subterms of n-grams

I am doing topic discovery on a large corpus. Reading here and there I found some papers saying that in case of big, sparse document x term matrices is better to create first a term x term similarity ...
0
votes
0answers
21 views

Clustering/Similarity between drivers

I have a dataset that contains initial and ending points of car trips: ...
1
vote
0answers
19 views

Is it possible to evaluate similarity (as in Kolmogorov-Smirnov test) between two sample vectors using copulas?

Consider the Two-sample Kolmogorov-Smirnov test: given 2 vectors of samples $X$ and $Y$ it is possible to test whether they are drawn from the same continuous distribution by calling (in R, for ...
2
votes
1answer
100 views

What GLM family and link function to use with similarity index as response variable?

I need to model an unusual response variable (at least to the best of my knowledge) that is: similarity (estimated by Morisita-Horn index) in species composition between pairs of sites. My response ...
0
votes
1answer
10 views

String similarity measure for pitch contour similarity (Parson code)

I have a dataset with short melodies hummed by experimental subjects. Each melody consists of a variable number of notes (10 to 20 notes) and is coded by the pitch levels of the notes (automatically ...
0
votes
0answers
13 views

Using the appropriate method for measuring the similarity of 1-D discrete time series data

I have a data of a 1-D time series data (more than 4000 points), they have fixed spacing, sample at fixed frequency. My task is to 'extract' the part in this data that match/look similar to my input ...
1
vote
0answers
18 views

R daisy - Gower distance, different values with different number of observations [duplicate]

I have a mixed data set where I want to compute distances between observations with Gower in R (daisy function). When I compute distances between different number of observations, the distances seem ...
1
vote
0answers
11 views

What can I use to compute a similarity (or diversity) index for a sample with “multidimensional” attributes?

Current problem: We have a batch of $n$ items for which we capture their details with $m$ attributes. It could look something like this: The goal is to compute an "index" that says how "similar" this ...
0
votes
0answers
6 views

How to comparing the similarity between two predictions with 2 fixed+1 alternative outcomes?

I want to compare the similarity of two diseases to Parkinson's disease (PD) with the same predictors. My first instinct is to run a multinomial regression with two fixed ends (healthy control and ...
0
votes
0answers
28 views

Similarity measure before and after dimensionality reduction or clustering

I have a dataset with 500 000 samples, each sample contains 30 features. The values of the features are in the range 0.0 to 1.0. ...
2
votes
1answer
50 views

Distance metric with characteristics of cosine and Manhattan

I'm working on a project where I want to find similarities between groups of events. So far I have expressed groups of events as vectors of event counts and computing similarities between them. I'm ...
0
votes
0answers
18 views

Mahalanobis: Covariance or correlation? [duplicate]

Is the Mahalanobis distance using correlations or covariances (among two vectors) to determine the similarity? I know that both are quite the same and differ only by a division of a standard deviation ...
1
vote
1answer
21 views

Clustering algorithm for similarity measure on quantitative values

I have a big set of subsets of 2 or more documents (maximum 10). The documents from each subset should describe the same product. One document contains pairs attribute - value: ...
0
votes
0answers
11 views

Correlation-like characteristic for ordinal feature with only three values

I have tabular data with two variables: one of them is regular continuous (float) variable and the second one is ordinal with three values only. I want to measure their dependency. (If it can help, I ...
0
votes
1answer
46 views

Calculating similarity between two lists: high cosine similarity, but high RMSE

I want to see how similar two datasets are, as a way to justify that they can be used in similar contexts. In practice one dataset contains manually calculated data, and the other automatically ...
0
votes
1answer
81 views

Function of Matrix Profile Index and Joins

I would like to find similar motifs in a time-series using Matrix Profile, but by selecting the sub-sequence manually, instead of doing dimension reduction. Matrix Profile From what I understand, ...
0
votes
0answers
28 views

Different backgrounds, different similarities with ecospat (R language)

I beg all of you your excuses, because the stupid question. I am exploring niche conservatism in five species of Ericaceae. I created two backgrounds sets in order to explore niche similarity (using R ...
1
vote
1answer
205 views

How to use Matrix Profile for dimension reduction and clustering

Matrix Profile (MP) has been used for clustering time-series segments. In the slideshow tutorial featured on the MP website, they use a figure to demonstrate projecting segment similarity onto an M-...
0
votes
0answers
16 views

Is there a method that's better than clustermaps for clustering samples which have a n of variables but only two possible outcomes?

For each variable there are two possible outcomes 1 or 0 (True or False): It shows the appearance of particular species on different localities. Mine goal is to cluster/classify similar localities ...
0
votes
1answer
75 views

Text Similarity - Cosine - Control. Suggestion to another / better method?

I would like to ask you, if anybody could check my code, because it was behaving weird - not working, giving me errors to suddenly working without changing anything - the code will be at the bottom. ...
0
votes
1answer
31 views

Similarity index between two texts Ask Question

I'm trying to compare two vectors in a small NLP project using Python. Code doesn't make any difference since I'm using scikit-learn, but my doubts are about my calculations. I have a query vector ...
0
votes
1answer
30 views

Similarity between 2 profiles (observations). Is it possible to generate a % similarity?

I have multiple profiles for 10 different people. Each person has been measured for 5 different continuous variables of different magnitudes. So my dataframe is 10x5 where each row represents a person ...
0
votes
0answers
19 views

Somers' D deteriorating for more granular rankings

Background I have 30 observations and I want to compare how much experts and model agree with each other. For Model and Expert I have slots which are $\in [1,4]$ and ranking which are $\in [1,30]$. ...
1
vote
0answers
48 views

Similarities / suggestions of multi-attribute entities

Dataset: I have 200 000 entites, these have zero or more of 1000 attributes (commonly 1-20 of them) The attibute relation is weighted 1-3 Its stored in a relational database modeles as Entity, ...
2
votes
0answers
69 views

Similarity between Train and Test data sets

I have multiclass classification dataset and I am using Deep nets for the classification task. To explain the problem, let's assume that I have 5 classes to classify. No matter what I try, be it ...
-2
votes
1answer
53 views

Find similar images in a dataset without labels [closed]

I have a set of grayscale images, some of them are transformed of the other images. For example in 10 images, image 2 is the same as image 8 but rotated, and image 4 is the same as image 7 but ...
0
votes
1answer
53 views

Clustering visited vertex in a graph

I would like to know can anyone introduce me an algorithm to cluster multiple sequence of visited vertices in a graph. For example assume your graph has 6 nodes. following might be 3 different ...
0
votes
0answers
166 views

Type of Recommendation engine algorithm to use

I'm planning to build a recommendation engine for a market place. The market place has User1,User2,,,,,,,UserN (with new users joining all the time). Each user has city and state information. Market ...
-2
votes
1answer
19 views

R Find similiar combinations of entries in vectors

I have a problem where I have no idea how to solve it. I have many character vectors which entries are mostly containing 1-4 words. Here´s some sample data: ...
1
vote
0answers
24 views

Predicting categories based on a similarity matrix

I am looking for some help in organising some analyses. I will describe what I am trying to do with a fictional example and then talk about some of the things I've thought about already. Example I ...
0
votes
2answers
63 views

Similarity function for machine learning training examples

I have a bunch of training examples (which are time series segments) that are used to train an algorithm. For any new example that is now presented to the algorithm for classification, i want to the ...
0
votes
0answers
23 views

Quantifying similarity of two objects in a population on certain characteristics

I have a set of objects (these are businesses) and for these businesses I have data on a number of characteristics. To keep this simple, we will consider these to be number of staff number of ...
0
votes
0answers
23 views

Positional similarity

What is a good positional similarity measure? Bottom line - words, which are next to each other should be ranked higher (no semantic or fuzzy matching needed). I want to use this in addition to TF/IDF-...
0
votes
0answers
37 views

Evaluate the similarity of distributions that are a function of a continuous parameter

Consider a continuous bound independent variable $X$ (for context, the altitude angle measured on a telescope) for which one can measure zero or more values of a continuous dependent variable $Y(X_i)_{...
1
vote
1answer
27 views

similarity distance when weight should change

I am trying to find a similarity measure for a very specific problem. I have a list of cities that have some characteritisques. For example bilingual school, firemen stations, etc. These cities are ...
2
votes
0answers
90 views

Distance vs Dissimilarity measure

I am reading up on distance and dissimilarity measures for my class on natural language processing and could not understand this slide. Why does the dissimilarity measure not satisfy item 3 ? What ...
1
vote
0answers
80 views

“Fuzzy” but simple and elegant similarity measure for time series

I am looking for a similarity measure between series which exploits the time dimension (rather than just its "mapping" aspect), and was unsatisfied with everything I found. First an example: assume I ...