Questions tagged [jaccard-similarity]

Jaccard similarity (or jaccard coefficient) is a similarity function for computing the similarity between two sets

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Can I use the masi distance to compare coders where the text document is not exactly the same?

It seems that Passoneau's MASI distance https://academiccommons.columbia.edu/doi/10.7916/D8PV6TTT was meant to grade different judges on the same document. Could I use this distance method to compare ...
0
votes
0answers
15 views

Is the Bray-Curtis dissimilarity/similarity independent of richness/diversity?

I was told about potential issues with the Bray-Curtis dissimilarity/similarity measure. Specifically, I have been asked if Bray-Curtis is independent of richness/diversity. I am unsure how to answer ...
0
votes
0answers
9 views

Find the hierarchical clustering based on Jaccard Score

I have an excel sheet where I have String details in one column, substring details in one column and jaccard_coefficient in one column respectively :- ...
1
vote
0answers
9 views

Clustering of individuals based on their time spent at various points of interest

Consider a set of individuals, denoted by $\mathcal{I}$, and a set of points of interest (POIs), denoted by $\mathcal{J}$ (with $|\mathcal{I} | \approx 10^4$ and $|\mathcal{J} | \leq 50$). For each ...
0
votes
0answers
20 views

How can I get the similarity and dissimilarity of two real numbers?

I would like to calculate the similarity and/or dissimilarity between two real numbers. The challenge is, we can have multiple types of real numbers. Say for example, Case-1: ...
0
votes
0answers
14 views

Choosing a Clustering Method Using Similarity Matrix of Binary Data (in SPSS via Hierarchical: Open to SAS, R, or Others)

I am working on a (somewhat exploratory) clustering of linguistic attributes for data we are working with. In an experiment, we designed prompts to elicit responses, hoping to prompt some sarcasm use. ...
0
votes
0answers
12 views

Measuring similarity between two objects across multiple categories

I'm constructing a crude measure to gauge the similarity between any pairs of objects across multiple dimensions (or categories (for example, they can be percentages of GDP across economic sectors or ...
1
vote
0answers
19 views

How to determine agreement between two sets?

I have two sets of Ids both drawn from the same background. I'd like to know the appropriate way to compare their similarity. Set A and B are often significantly different in length eg. ...
0
votes
1answer
43 views

Are Jaccard score and F1 score monotonically related?

I have compared the rankings obtained by comparing 10+ classifiers with this two metrics: Jaccard score F1 score They show a perfect correlation. This results holds on 50+ datasets. When comparing ...
0
votes
0answers
10 views

Weighted similarity coefficient for binary data?

I would like to evaluate the accuracy of land degradation maps I have estimated using remote sensing compared against observed field land degradation maps. The maps are classified in a binary way and ...
1
vote
0answers
15 views

Clustering - Distance Metric for Comparing Short Lists of Terms (non-repeating, no frequency)

Clustering involves using some distance or similarity metric. What is the best way to score the similarity of these small sets of words? Criteria: These are technical terms which are extracted from ...
0
votes
1answer
55 views

String distance equations for fuzzy-matching person names

A reproducible example with a small bit of R code is available in this stackoverflow post (link so I dont need to re-type out the code). The fuzzytext library in R ...
2
votes
2answers
667 views

Is Jaccard similarity/distance suitable for non-binary, quantitative data?

I have a dataset with each row a country and 10 columns with numerical features like GDP,Electrcity consumption, GNI etc. I am trying to use distance metrics to find similarity between the countries ...
0
votes
0answers
63 views

What's the advantage of cosine distance over Jaccard distance for text document similarity

We usually use cosine distance as the similarity measure for text document. However, Jaccard distance also somehow make sense to me. My question is for text document, is there any advantage of using ...
2
votes
1answer
56 views

Difference between an interaction and combining two variables into one with a similarity measure?

Conceptually, what would be the difference between interacting two variables in a regression and combining these two variables into one via a similarity measure? I'm looking at top managers and boards ...
0
votes
0answers
140 views

Performance of model decreases when calculating Euclidean distance between vectors with TF.IDF weigths compared to TF weights

My problem in short: I use Jaccard similarity, cosine similarity and euclidean distance to compute a similarity between documents. The documents consists of either the words,hashtags or combination of ...
0
votes
0answers
114 views

Jaccard/binary (dis)similarity calculation to multidimensional scaling analysis

I have n N x n dataset of features (n) and subjects (N) in which I am attempting to cluster into a lower-dimensional space via multi-dimensional scaling. I'm confused about which MDS setup I need to ...
0
votes
0answers
66 views

Is there a way calculate a gradient of dissimilarity across multiple sites using presence/absence data?

I’ve been trying to come up with a metric for species (dis)similarity for fish in a series based on presence or absence data that gives a value of 1 or 0 for each lake. From my research, it seems like ...
0
votes
1answer
103 views

How to create a “similarity index” on a set of variables

I have a set of 12 variables that describe the dwelling characteristics. 10 of them are dummies (i.e questions like "does the dwelling have a bath?" or "is there crime in the area?"), while one is a 4-...
0
votes
1answer
53 views

distance metric for student course schedules

I'm doing an exploratory clustering analysis of student course schedules at a college. Interpretability by humans is paramount: we're trying to inform future research questions and possibly ...
1
vote
1answer
646 views

Jaccard Similarity with Binary Data

I have 5400 rows of data and 3211 columns of attributes. The first 4 columns are ID/Name/ParentID/ObjectType - the rest of the 3207 columns are the attributes that are to be used for similarity ...
0
votes
1answer
227 views

Similarity between n >> 2 binary vectors

I have $n$ binary vectors $x_i, ..., x_n$ of length $d$. I wish to compute a similarity measure between all $n$ of them. My initial thought was to use the Jaccard Index, since each binary element $x_{...
-1
votes
1answer
119 views

jaccard distance if one samples is a subset

I have a very basic question but I cannot find an answer (especially for a clustering situation). I am trying to do hierarchical clustering of samples using jaccard distances. One sample contains ...
1
vote
2answers
105 views

Addressing “Unbalanced Features” or Feature Taxonomy for Nearest Neighbor / Similarity Calculations

The main question is how to address an imbalance in representation of feature "sets" when calculating similarity. I'll motivate with an example scenario: Suppose we have objects described by a binary ...
2
votes
0answers
103 views

Unique Jaccardian Distance?

Hi I have a dataset that could be simulated using: ...
0
votes
3answers
3k views

Can I use Manhattan distance on binary data for hierarchical clustering?

I understand that classically Jaccard and Hamming work best with binary data, but is there anything specifically wrong with using a Manhattan distance instead with the complete linkage function?
3
votes
1answer
421 views

Estimate Jaccard similarity based on a sample

The Jaccard similarity of two sets, $A$ and $B$, is defined as: $Jaccard(A,B)=\frac{A\cap{B}}{A\cup{B}}$. Say that I only have a sample of $P\%$ of each of the sets: $A'$ and $B'$. What would be a ...
0
votes
0answers
255 views

Measure similar to Jaccard index but comparing more than two groups at a time

I am interested in looking at the amount of sharing between several samples (each represented as a vector of character strings), and to do pairwise comparisions, I have been using Jaccard indices. Is ...
1
vote
1answer
5k views

Calculating Jaccard similarity in R: all duo combinations for multiple groups and creating a matrix or cluster plotting with the index [closed]

I have been trying to compute Jaccard similarity index for all possible duo combinations for 7 communities and to create a matrix, or preferably Cluster plotting with the similarity index. There are ...
1
vote
0answers
566 views

How do I calculate the Jaccard distance for normal distributions?

I'm trying to replicate the results in this paper. I'm stuck at this part: specifically, calculate Jaccard intersection of pdata1 & pdata2 Is there some neat ...
4
votes
1answer
2k views

What is the significance of the Jaccard similarity score?

I understand how to calculate the jaccard similarity , but never quite understood the logic behind why are we calculating it. How does it show the similarity between two sets? What relation exactly ...
1
vote
0answers
72 views

Correlation of continuous-time series

I have two continuous time series of discrete events. The data from them are timestamps of occurences, e.g. ...
1
vote
1answer
74 views

Measuring similarity of observations (non numeric)

I have a dataset of format : day,measurement1,measurement2 1,a,b 1,a,c 1,f,s 2,a,b 2,a,c 2,f,g 3,a,d 3,a,q 3,f,s In this example day1 is more similar with day2 ...
0
votes
0answers
232 views

Calculating Similarity between two multiset

I need to find similarity between two multiset. For example, $$A = \{a, a, b, c, d\}$$ $$B = \{a, b, b, d\}$$ I know that I can use Jaccard Index but it gives same weight for each element. As ...
1
vote
0answers
432 views

Dummy data clustering: OK to apply Euclidian distance hierarchical clustering on Jaccard similarity matrix?

I created dummy variables (binary data) from categorical variables where I want to partition N subjects into multiple classes by some clustering method. I created a Jaccard similarity index matrix for ...
0
votes
0answers
47 views

Similarity between bar plots (counts of categories)?

What are the best ways to compare (calculate similarity) between two bar plots? For example: ...
7
votes
3answers
5k views

Similarity measures for more than 2 variables

If I have two binary variables, I can determine the similarity of these variables quite easily with different similarity measures, e.g. with the Jaccard similarity measure: $J = \frac{M_{11}}{M_{01} +...
1
vote
1answer
457 views

Interpretation of Jaccard similarity for binary networks

Given two undirected networks with the same amount of nodes defined by $A_{ij}^1$ and $A_{ij}^2$, we take the unique unordered pairs $(i, j)$. After sorting them, we then have two binary vectors $\vec{...
1
vote
0answers
52 views

How to statistically validate that one clustering technique is more effective than other on Binary data

Problem statement - Idea is to cluster students based on their assessment pattern and score. Data = I have a data set that contains Binary data of students answers(Correct/incorrect) which i have ...
-1
votes
1answer
59 views

How to form clusters when features are sets of string objects

I am building a movie recommender. But unlike common recommender from movielens data, features of my movies are sets like actor list, genre list, list of producers and writers etc. I know I can use ...
1
vote
2answers
381 views

Detecting misclassified data points using distance matrix

I have a data set of around 65,000 items from a public database that have each been assigned to one of around 1,800 classes, but unfortunately I am aware that quite a few are misclassified. My goals ...
5
votes
2answers
3k views

Jaccard similarity coefficient vs. Point-wise mutual information coefficient

Can you explain the difference between the Jaccard similarity coefficient and the pointwise mutual information (PMI) measure? It would be great if you could add a few examples.
7
votes
2answers
10k views

Accuracy vs Jaccard for multiclass problem

TL;DR For a multiclass problem, is Jaccard score the same as accuracy? Update March 29, 2019 The wrong implementation in scikit-learn is now fixed with pull request #13151. Hooray! P.S. The lesson ...
1
vote
0answers
155 views

Threshold value for nodes similarity

I am working on an algorithm for community detection on complex systems, where those systems are modeled as graphs. To form community i need to find similarity between nodes. I am using Jaccard ...
0
votes
0answers
461 views

Jaccard coefficient (0011 and 1100 most similar?)

I try to compare different binary objects, lets say: ...
2
votes
2answers
2k views

How to compare a Ranked List?

I have 50 products. For each product, I identify 3 related products (1 top related, 2 (partially related, least related)) using similarity measures. I want to compare the ranked list generated by my ...
0
votes
2answers
859 views

Jaccard similarity of sets with approximate element matching

Consider two sets like this: A = {'hello', 'world'} and B = {'Hello', 'helo', 'goodbye'}. I want to measure the similarity between these two sets. There are two levels to this problem: Comparing ...
2
votes
1answer
3k views

NMDS from Jaccard and Bray-Curtis identical. Is that a bad thing?

The dataset is ecological (species abundance), where I am calculating distances between 51 sampling sites, based on abundances of 3200 species. The species abundance values go up to 2700, and the rest ...
2
votes
1answer
7k views

Jaccard distance vs Levenshtein distance for fuzzy matching

My data is similar to the following data, but far bigger and more complex. Apple Banana Those fruits Tomato Cocumber These vegetables I would like to get the ...
15
votes
1answer
16k views

What are the difference between Dice, Jaccard, and overlap coefficients? [closed]

I come across three different statistical measures to compare two sets, in particular to segmentation on images (e.g., comparing the similarity between the ground truth and the segmented result). ...