Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [similarities]

Measures of closeness between distributions, clusterings, data sets or other objects.

0
votes
1answer
35 views

Comparison of two normal distribution

I have two normally distributed samples. I want to know how close or similar it is. I tried few methods to find the similarity, like z-score and bhattacharyya distance. Bhattacharyya distance didn't ...
1
vote
1answer
22 views

Calculate Earth Mover's Distance for two grayscale images

I am trying to calculate EMD (a.k.a. Wasserstein Distance) for these two grayscale (299x299) images/heatmaps: Right now, I am calculating the histogram/distribution of both images. The histograms ...
2
votes
1answer
35 views

Statistically compare similarity between images

I have two images/heatmaps (2d matrix) of identical size. I need to statistically compare the similarity between the two. With 'similarity', I mean that high and low values of one image appear in ...
0
votes
0answers
12 views

How to get similarity matrix from a random forest model? [on hold]

I have trained a randomforestclassifier on a dataset with 652 samples. I have achieved 89.6% accuracy. Now I want to extract the similarity matrix from the trained classifer and do some clustering. ...
0
votes
1answer
11 views

similarity function for keywords that often occur together

I'm working on a project where I have to develop a machine learning solution. I should mention that I'm not very experienced in ML yet and that I'm not even sure if the term "similarity function" is ...
0
votes
0answers
9 views

Dimensionality of similarity matrix

Below is a screen shot of a paper. The authors take a data-set $E\in R^{nxm}$. Here $n$ is the number of observations/samples/patients and $m$ is the number of genes/features. Preprocessing ...
0
votes
1answer
11 views

What distance function to use for matching observations on a mix of ordinal, nominal and numeric variables?

I'm trying to match potential respondents to a random sample from the target population, and not sure whether any of the usual methods available out-of-the-box in e.g. R are right for the job. ...
0
votes
0answers
9 views

Term x term matrix for text clustering. What to do with subterms of n-grams

I am doing topic discovery on a large corpus. Reading here and there I found some papers saying that in case of big, sparse document x term matrices is better to create first a term x term similarity ...
0
votes
0answers
16 views

Clustering/Similarity between drivers

I have a dataset that contains initial and ending points of car trips: ...
1
vote
0answers
16 views

Is it possible to evaluate similarity (as in Kolmogorov-Smirnov test) between two sample vectors using copulas?

Consider the Two-sample Kolmogorov-Smirnov test: given 2 vectors of samples $X$ and $Y$ it is possible to test whether they are drawn from the same continuous distribution by calling (in R, for ...
2
votes
1answer
75 views

What GLM family and link function to use with similarity index as response variable?

I need to model an unusual response variable (at least to the best of my knowledge) that is: similarity (estimated by Morisita-Horn index) in species composition between pairs of sites. My response ...
0
votes
1answer
10 views

String similarity measure for pitch contour similarity (Parson code)

I have a dataset with short melodies hummed by experimental subjects. Each melody consists of a variable number of notes (10 to 20 notes) and is coded by the pitch levels of the notes (automatically ...
0
votes
0answers
12 views

Using the appropriate method for measuring the similarity of 1-D discrete time series data

I have a data of a 1-D time series data (more than 4000 points), they have fixed spacing, sample at fixed frequency. My task is to 'extract' the part in this data that match/look similar to my input ...
1
vote
0answers
15 views

R daisy - Gower distance, different values with different number of observations [duplicate]

I have a mixed data set where I want to compute distances between observations with Gower in R (daisy function). When I compute distances between different number of observations, the distances seem ...
1
vote
0answers
11 views

What can I use to compute a similarity (or diversity) index for a sample with “multidimensional” attributes?

Current problem: We have a batch of $n$ items for which we capture their details with $m$ attributes. It could look something like this: The goal is to compute an "index" that says how "similar" this ...
0
votes
0answers
6 views

How to comparing the similarity between two predictions with 2 fixed+1 alternative outcomes?

I want to compare the similarity of two diseases to Parkinson's disease (PD) with the same predictors. My first instinct is to run a multinomial regression with two fixed ends (healthy control and ...
0
votes
0answers
27 views

Similarity measure before and after dimensionality reduction or clustering

I have a dataset with 500 000 samples, each sample contains 30 features. The values of the features are in the range 0.0 to 1.0. ...
2
votes
1answer
44 views

Distance metric with characteristics of cosine and Manhattan

I'm working on a project where I want to find similarities between groups of events. So far I have expressed groups of events as vectors of event counts and computing similarities between them. I'm ...
0
votes
0answers
18 views

Mahalanobis: Covariance or correlation? [duplicate]

Is the Mahalanobis distance using correlations or covariances (among two vectors) to determine the similarity? I know that both are quite the same and differ only by a division of a standard deviation ...
1
vote
1answer
21 views

Clustering algorithm for similarity measure on quantitative values

I have a big set of subsets of 2 or more documents (maximum 10). The documents from each subset should describe the same product. One document contains pairs attribute - value: ...
0
votes
0answers
11 views

Correlation-like characteristic for ordinal feature with only three values

I have tabular data with two variables: one of them is regular continuous (float) variable and the second one is ordinal with three values only. I want to measure their dependency. (If it can help, I ...
0
votes
1answer
34 views

Calculating similarity between two lists: high cosine similarity, but high RMSE

I want to see how similar two datasets are, as a way to justify that they can be used in similar contexts. In practice one dataset contains manually calculated data, and the other automatically ...
0
votes
1answer
59 views

Function of Matrix Profile Index and Joins

I would like to find similar motifs in a time-series using Matrix Profile, but by selecting the sub-sequence manually, instead of doing dimension reduction. Matrix Profile From what I understand, ...
0
votes
0answers
28 views

Different backgrounds, different similarities with ecospat (R language)

I beg all of you your excuses, because the stupid question. I am exploring niche conservatism in five species of Ericaceae. I created two backgrounds sets in order to explore niche similarity (using R ...
1
vote
1answer
119 views

How to use Matrix Profile for dimension reduction and clustering

Matrix Profile (MP) has been used for clustering time-series segments. In the slideshow tutorial featured on the MP website, they use a figure to demonstrate projecting segment similarity onto an M-...
0
votes
0answers
14 views

Is there a method that's better than clustermaps for clustering samples which have a n of variables but only two possible outcomes?

For each variable there are two possible outcomes 1 or 0 (True or False): It shows the appearance of particular species on different localities. Mine goal is to cluster/classify similar localities ...
0
votes
1answer
54 views

Text Similarity - Cosine - Control. Suggestion to another / better method?

I would like to ask you, if anybody could check my code, because it was behaving weird - not working, giving me errors to suddenly working without changing anything - the code will be at the bottom. ...
0
votes
0answers
44 views

Fuzzy String Matching and Word Length

I'm attempting to calculate how likely it is that two words are the "same", but misspelled. I've narrowed it down to a few algorithms that will give me a similarity metric, but the problem is that ...
0
votes
1answer
30 views

Similarity index between two texts Ask Question

I'm trying to compare two vectors in a small NLP project using Python. Code doesn't make any difference since I'm using scikit-learn, but my doubts are about my calculations. I have a query vector ...
0
votes
1answer
27 views

Similarity between 2 profiles (observations). Is it possible to generate a % similarity?

I have multiple profiles for 10 different people. Each person has been measured for 5 different continuous variables of different magnitudes. So my dataframe is 10x5 where each row represents a person ...
0
votes
0answers
17 views

Somers' D deteriorating for more granular rankings

Background I have 30 observations and I want to compare how much experts and model agree with each other. For Model and Expert I have slots which are $\in [1,4]$ and ranking which are $\in [1,30]$. ...
1
vote
0answers
38 views

Similarities / suggestions of multi-attribute entities

Dataset: I have 200 000 entites, these have zero or more of 1000 attributes (commonly 1-20 of them) The attibute relation is weighted 1-3 Its stored in a relational database modeles as Entity, ...
2
votes
0answers
59 views

Similarity between Train and Test data sets

I have multiclass classification dataset and I am using Deep nets for the classification task. To explain the problem, let's assume that I have 5 classes to classify. No matter what I try, be it ...
-2
votes
1answer
50 views

Find similar images in a dataset without labels [closed]

I have a set of grayscale images, some of them are transformed of the other images. For example in 10 images, image 2 is the same as image 8 but rotated, and image 4 is the same as image 7 but ...
0
votes
1answer
52 views

Clustering visited vertex in a graph

I would like to know can anyone introduce me an algorithm to cluster multiple sequence of visited vertices in a graph. For example assume your graph has 6 nodes. following might be 3 different ...
0
votes
0answers
144 views

Type of Recommendation engine algorithm to use

I'm planning to build a recommendation engine for a market place. The market place has User1,User2,,,,,,,UserN (with new users joining all the time). Each user has city and state information. Market ...
-2
votes
1answer
19 views

R Find similiar combinations of entries in vectors

I have a problem where I have no idea how to solve it. I have many character vectors which entries are mostly containing 1-4 words. Here´s some sample data: ...
1
vote
0answers
21 views

Predicting categories based on a similarity matrix

I am looking for some help in organising some analyses. I will describe what I am trying to do with a fictional example and then talk about some of the things I've thought about already. Example I ...
0
votes
2answers
58 views

Similarity function for machine learning training examples

I have a bunch of training examples (which are time series segments) that are used to train an algorithm. For any new example that is now presented to the algorithm for classification, i want to the ...
0
votes
0answers
22 views

Quantifying similarity of two objects in a population on certain characteristics

I have a set of objects (these are businesses) and for these businesses I have data on a number of characteristics. To keep this simple, we will consider these to be number of staff number of ...
0
votes
0answers
21 views

Positional similarity

What is a good positional similarity measure? Bottom line - words, which are next to each other should be ranked higher (no semantic or fuzzy matching needed). I want to use this in addition to TF/IDF-...
0
votes
0answers
36 views

Evaluate the similarity of distributions that are a function of a continuous parameter

Consider a continuous bound independent variable $X$ (for context, the altitude angle measured on a telescope) for which one can measure zero or more values of a continuous dependent variable $Y(X_i)_{...
1
vote
1answer
23 views

similarity distance when weight should change

I am trying to find a similarity measure for a very specific problem. I have a list of cities that have some characteritisques. For example bilingual school, firemen stations, etc. These cities are ...
0
votes
0answers
73 views

On the right dissimilarity matrix for discrete cluster analysis

Currently I'm having some trouble on deciding the right dissimilarity matrix for my data. After reading some previously asked questions here, I'm even left more confused. So, here is the deal. My data ...
2
votes
0answers
53 views

Distance vs Dissimilarity measure

I am reading up on distance and dissimilarity measures for my class on natural language processing and could not understand this slide. Why does the dissimilarity measure not satisfy item 3 ? What ...
0
votes
0answers
31 views

Using some kind of similarity measure to predict target variable from historical data

Let me first describe the data I have: I have aggregated CAN bus data (1 second windows, from the original 50 ms values) from 6 forklifts, and I also have measurements for these 6 trucks which ...
1
vote
0answers
67 views

“Fuzzy” but simple and elegant similarity measure for time series

I am looking for a similarity measure between series which exploits the time dimension (rather than just its "mapping" aspect), and was unsatisfied with everything I found. First an example: assume I ...
0
votes
1answer
261 views

Best similarity measure for binary wine data

I'm creating a dataset of wine grape varieties and their associated flavors/aromas. Here's a schematic of the data: ...
2
votes
1answer
67 views

What is a “Nominal Type Histogram” and “Shuffling Invariant Property”?

I'm reading this paper about comparing histograms ["Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions" by Sung-Hyuk Cha] that writes in the introduction: ...
12
votes
2answers
238 views

Does Mercer's theorem work in reverse?

A colleague has a function $s$ and for our purposes it is a black-box. The function measures the similarity $s(a,b)$ of two objects. We know for sure that $s$ has these properties: The similarity ...