Questions tagged [jaccard-similarity]
Jaccard similarity (or jaccard coefficient) is a similarity function for computing the similarity between two sets
0
votes
0answers
12 views
Set Similarity Measure that accounts for size
Suppose I have two sets, A and B where 10|A| = |B| and A is a subset of B. For the case where |A| = 1 the Jaccard similarity of the sets will be 0.1. For the case where |A| = 100 the Jaccard ...
1
vote
1answer
70 views
Jaccard Similarity with Binary Data
I have 5400 rows of data and 3211 columns of attributes.
The first 4 columns are ID/Name/ParentID/ObjectType - the rest of the 3207 columns are the attributes that are to be used for similarity ...
0
votes
1answer
21 views
Similarity between n >> 2 binary vectors
I have $n$ binary vectors $x_i, ..., x_n$ of length $d$. I wish to compute
a similarity measure between all $n$ of them. My initial thought was to use
the Jaccard Index, since each binary element $x_{...
0
votes
0answers
217 views
Jaccard Index for Binary Data in R using dist function
I have a presence/absence table of 0s and 1s and I would like to cluster this data. I want to create a pairwise matrix using R's dist function which has a binary ...
-1
votes
1answer
49 views
jaccard distance if one samples is a subset
I have a very basic question but I cannot find an answer (especially for a clustering situation). I am trying to do hierarchical clustering of samples using jaccard distances. One sample contains ...
0
votes
1answer
45 views
Addressing “Unbalanced Features” or Feature Taxonomy for Nearest Neighbor / Similarity Calculations
The main question is how to address an imbalance in representation of feature "sets" when calculating similarity. I'll motivate with an example scenario:
Suppose we have objects described by a binary ...
1
vote
0answers
56 views
Jaccard Similarity in R Input Prep [closed]
Could I get assistance generating dynamic comparison vectors to use in the Jaccard function in R? For instance, if I have two sets:
"a" "b" "c"
and
"b" "c" "f"
the union vector would be a,b,c,f ...
1
vote
0answers
55 views
Downsampling features: how to select the most optimal features to recapitulate clustering [closed]
I've performed single cell analysis in which each gene represents a feature to cluster upon; there are about 20,000 genes expressed across all the cells in the dataset. I use the top 1500 or so ...
0
votes
0answers
218 views
Jaccard distance for sets of different sizes
Is the Jaccard distance still a metric if applied to sets of different sizes? Or does it require the input sets are of the same length?
0
votes
0answers
88 views
Jaccard Similarity for matrices
I want to use the jaccard similarity to measure the quality of my binary segmentation. I found a function in the library sklearn.metrics called jaccard_similarity_score and I found in another forum ...
1
vote
0answers
79 views
0
votes
3answers
1k views
Can I use Manhattan distance on binary data for hierarchical clustering?
I understand that classically Jaccard and Hamming work best with binary data, but is there anything specifically wrong with using a Manhattan distance instead with the complete linkage function?
3
votes
0answers
227 views
Estimate Jaccard similarity based on a sample
The Jaccard similarity of two sets, $A$ and $B$, is defined as: $Jaccard(A,B)=\frac{A\cap{B}}{A\cup{B}}$.
Say that I only have a sample of $P\%$ of each of the sets: $A'$ and $B'$. What would be a ...
0
votes
0answers
213 views
Measure similar to Jaccard index but comparing more than two groups at a time
I am interested in looking at the amount of sharing between several samples (each represented as a vector of character strings), and to do pairwise comparisions, I have been using Jaccard indices. Is ...
0
votes
1answer
2k views
Calculating Jaccard similarity in R: all duo combinations for multiple groups and creating a matrix or cluster plotting with the index [closed]
I have been trying to compute Jaccard similarity index for all possible duo combinations for 7 communities and to create a matrix, or preferably Cluster plotting with the similarity index.
There are ...
1
vote
0answers
283 views
How do I calculate the Jaccard distance for normal distributions?
I'm trying to replicate the results in this paper. I'm stuck at this part: specifically,
calculate Jaccard intersection of pdata1 & pdata2
Is there some neat ...
4
votes
1answer
958 views
What is the significance of the Jaccard similarity score?
I understand how to calculate the jaccard similarity , but never quite understood the logic behind why are we calculating it. How does it show the similarity between two sets? What relation exactly ...
1
vote
0answers
49 views
Correlation of continuous-time series
I have two continuous time series of discrete events. The data from them are timestamps of occurences, e.g.
...
1
vote
1answer
35 views
Measuring simmilarity of observations (non numeric)
I have a dataset of format :
day,measurement1,measurement2
1,a,b
1,a,c
1,f,s
2,a,b
2,a,c
2,f,g
3,a,d
3,a,q
3,f,s
In this example day1 is more similar with day2 ...
0
votes
0answers
131 views
Calculating Similarity between two multiset
I need to find similarity between two multiset. For example,
$$A = \{a, a, b, c, d\}$$
$$B = \{a, b, b, d\}$$
I know that I can use Jaccard Index but it gives same weight for each element. As ...
1
vote
0answers
293 views
Dummy data clustering: OK to apply Euclidian distance hierarchical clustering on Jaccard similarity matrix?
I created dummy variables (binary data) from categorical variables where I want to partition N subjects into multiple classes by some clustering method. I created a Jaccard similarity index matrix for ...
0
votes
0answers
31 views
Similarity between bar plots (counts of categories)?
What are the best ways to compare (calculate similarity) between two bar plots? For example:
...
5
votes
3answers
2k views
Similarity measures for more than 2 variables
If I have two binary variables, I can determine the similarity of these variables quite easily with different similarity measures, e.g. with the Jaccard similarity measure:
$J = \frac{M_{11}}{M_{01} +...
1
vote
1answer
368 views
Interpretation of Jaccard similarity for binary networks
Given two undirected networks with the same amount of nodes defined by $A_{ij}^1$ and $A_{ij}^2$, we take the unique unordered pairs $(i, j)$. After sorting them, we then have two binary vectors $\vec{...
1
vote
0answers
34 views
How to statistically validate that one clustering technique is more effective than other on Binary data
Problem statement - Idea is to cluster students based on their assessment pattern and score.
Data = I have a data set that contains Binary data of students answers(Correct/incorrect) which i have ...
-1
votes
1answer
45 views
How to form clusters when features are sets of string objects
I am building a movie recommender. But unlike common recommender from movielens data, features of my movies are sets like actor list, genre list, list of producers and writers etc.
I know I can use ...
1
vote
2answers
226 views
Detecting misclassified data points using distance matrix
I have a data set of around 65,000 items from a public database that have each been assigned to one of around 1,800 classes, but unfortunately I am aware that quite a few are misclassified. My goals ...
4
votes
2answers
2k views
Jaccard similarity coefficient vs. Point-wise mutual information coefficient
Can you explain the difference between the Jaccard similarity coefficient and the pointwise mutual information (PMI) measure? It would be great if you could add a few examples.
3
votes
2answers
5k views
Accuracy vs Jaccard for multiclass problem
TL;DR For a multiclass problem, is Jaccard score the same as accuracy?
Update March 29, 2019
The wrong implementation in scikit-learn is now fixed with pull request #13151. Hooray!
P.S. The lesson ...
1
vote
0answers
138 views
Threshold value for nodes similarity
I am working on an algorithm for community detection on complex systems, where those systems are modeled as graphs. To form community i need to find similarity between nodes.
I am using Jaccard ...
0
votes
0answers
333 views
Jaccard coefficient (0011 and 1100 most similar?)
I try to compare different binary objects, lets say:
...
1
vote
1answer
507 views
How to compare a Ranked List?
I have 50 products. For each product, I identify 3 related products (1 top related, 2 (partially related, least related)) using similarity measures. I want to compare the ranked list generated by my ...
0
votes
2answers
553 views
Jaccard similarity of sets with approximate element matching
Consider two sets like this:
A = {'hello', 'world'} and B = {'Hello', 'helo', 'goodbye'}.
I want to measure the similarity between these two sets. There are two levels to this problem:
Comparing ...
2
votes
1answer
2k views
NMDS from Jaccard and Bray-Curtis identical. Is that a bad thing?
The dataset is ecological (species abundance), where I am calculating distances between 51 sampling sites, based on abundances of 3200 species. The species abundance values go up to 2700, and the rest ...
2
votes
1answer
5k views
Jaccard distance vs Levenshtein distance for fuzzy matching
My data is similar to the following data, but far bigger and more complex.
Apple
Banana
Those fruits
Tomato
Cocumber
These vegetables
I would like to get the ...
7
votes
1answer
6k views
What are the difference between Dice, Jaccard, and overlap coefficients? [closed]
I come across three different statistical measures to compare two sets, in particular to segmentation on images (e.g., comparing the similarity between the ground truth and the segmented result).
...
-1
votes
1answer
35 views
I want to find similarity for 20 organisms on 25 parameters? How to do this ? Any software is available for this? [closed]
How to find similarity for 20 organisms on 25 parameters? How to do this ? Any software available for this?
1
vote
0answers
216 views
Find similarity score between multiple items based on tags assigned to them
I am working for a client who is in e-commerce. They have thousands of offerings and each has its own set of tags. Now, they want to find similar designs dynamically based on the item selected by the ...
4
votes
0answers
270 views
Statistical Interpretation of Average Pairwise Similarity
I have assembled binary vectors (0/1 for all elements and equal weight and arranged in time order) that have been separated into different cohorts where a unique event of interest occurs. I have ...
3
votes
0answers
105 views
Similarity measures and document length
I have an application where I need to measure the similarity between the (TF-IDF?) representation of two documents: $\mathbf{a}$ and $\mathbf{b}$ while still taking the document length into account.
...
0
votes
0answers
168 views
Numerical Similarity Measures for Dirty Data
I need to compare two datasets to see if they are "similar" in terms of numerical similarity between given columns. The problem is, however, that I do not know the characteristics of the 2nd dataset. ...
2
votes
2answers
2k views
Significance Test for Jaccard Distance
I am looking for a significance test for the Jaccard Distance (JD).
As an example, I have two datasets as follows:
Baseline: $\left| A\bigcap B \right|=57;\ \left| A\bigcup B \right|=275\quad \...
2
votes
1answer
328 views
Is the Jaccard Index a function of the number of elements in a list?
I am trying to understand a research result. Imagine 10 different psychological questionnaires 1,2,3,...10 of different length (i.e. number of questions) that are all supposedly measuring the same ...
2
votes
0answers
418 views
Approach to find similarity between two different types of matrices
I'm using the movielens dataset to give recommendations to a user based on genre of the movies.
I have two matrices, one contains the genre the user likes. We are considering user3 for now
I also ...
0
votes
1answer
88 views
How to soften or mitigate vector similarity measure?
I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located.
I construct two sets of nine concentric and equidistant circles with centers ...
1
vote
1answer
331 views
Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions
I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources.
http://www2007.org/papers/paper570.pdf
https://chrisjmccormick.wordpress.com/...
2
votes
0answers
916 views
Similarity between sets with different size
is there a distance measure like jaccard for sets with different sizes?
For example A=['a','b','c'] and B=['a','d']
I would like to include the total intersection as well as the order.
The ...
0
votes
1answer
853 views
confusion matrix and Jaccard index computation in O(n) for cluster comparison
My problem:
I have two clustering (or partitions) of a set of numbers $\{1,...,n\}$, each with $k$ cluster (or subsets), that i have to compare and, to do that, i'd like to try some different indexes ...
1
vote
1answer
816 views
Apriori versus Jaccard Coeff for recommender-system
I developed a market basket program for a retailer. I used the pymining library association rules function. I assume this uses the Apriori algorithm. I got 340,000 rules for an input of 17,000 ...
6
votes
5answers
20k views
Jaccard similarity in R
I want to compare 2 vectors of length 43; they have values of 0 (not present) and 1 (present). I will refer to $M_{1,1}$ as situations in which both 1 are present, and $M_{1,0}$ and $M_{0,1}$ to ...
