Questions tagged [jaccard-similarity]

Jaccard similarity (or jaccard coefficient) is a similarity function for computing the similarity between two sets

43 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
339 views

Statistical Interpretation of Average Pairwise Similarity

I have assembled binary vectors (0/1 for all elements and equal weight and arranged in time order) that have been separated into different cohorts where a unique event of interest occurs. I have ...
3
votes
0answers
120 views

Similarity measures and document length

I have an application where I need to measure the similarity between the (TF-IDF?) representation of two documents: $\mathbf{a}$ and $\mathbf{b}$ while still taking the document length into account. ...
3
votes
1answer
518 views

Is the Jaccard Index a function of the number of elements in a list?

I am trying to understand a research result. Imagine 10 different psychological questionnaires 1,2,3,...10 of different length (i.e. number of questions) that are all supposedly measuring the same ...
3
votes
0answers
515 views

Approach to find similarity between two different types of matrices

I'm using the movielens dataset to give recommendations to a user based on genre of the movies. I have two matrices, one contains the genre the user likes. We are considering user3 for now I also ...
3
votes
0answers
1k views

Similarity between sets with different size

is there a distance measure like jaccard for sets with different sizes? For example A=['a','b','c'] and B=['a','d'] I would like to include the total intersection as well as the order. The ...
3
votes
0answers
2k views

should I count repeated items in jaccard similarity?

I have two lists: A and B where Size(A) = 4 and size(B) = 10. I would like to find the Jaccard similarity between these two lists. Suppose A = {Tom, George, John, Jennifer} and B={Tom, Jessica, ...
3
votes
0answers
604 views

A probability distribution model for Jaccard similarity

This is an obfuscated version of a real problem: Each day I speak with some number of (distinct) girls. I compute the Jaccard similarity index between two consecutive days: $$ J(t)=J(t,t-1)=\...
3
votes
1answer
437 views

Estimate Jaccard similarity based on a sample

The Jaccard similarity of two sets, $A$ and $B$, is defined as: $Jaccard(A,B)=\frac{A\cap{B}}{A\cup{B}}$. Say that I only have a sample of $P\%$ of each of the sets: $A'$ and $B'$. What would be a ...
2
votes
0answers
103 views

Unique Jaccardian Distance?

Hi I have a dataset that could be simulated using: ...
2
votes
1answer
396 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf https://chrisjmccormick.wordpress.com/...
2
votes
0answers
1k views

Cosine similarity between two types of user profiles

I have a movie dataset containing preference (ratings) of users on movies and also attributes of movies (genre, cast, director). I created two types of profiles for users based on the movie attributes ...
2
votes
0answers
1k views

Finding nearest neighbors using Jaccard distance for positive, real-valued vectors

Say we have $x_i, \ldots, x_n \in R ^ D$ with positive, real components and use Jaccard distance $$d(x_i, x_j) = 1 - \frac{\sum_{d = 1}^D\min(x_i^d, x_j^d)}{\sum_{d = 1}^D\max(x_i^d, x_j^d)}$$ to ...
2
votes
1answer
7k views

Jaccard distance vs Levenshtein distance for fuzzy matching

My data is similar to the following data, but far bigger and more complex. Apple Banana Those fruits Tomato Cocumber These vegetables I would like to get the ...
1
vote
0answers
10 views

Clustering of individuals based on their time spent at various points of interest

Consider a set of individuals, denoted by $\mathcal{I}$, and a set of points of interest (POIs), denoted by $\mathcal{J}$ (with $|\mathcal{I} | \approx 10^4$ and $|\mathcal{J} | \leq 50$). For each ...
1
vote
0answers
26 views

How to determine agreement between two sets?

I have two sets of Ids both drawn from the same background. I'd like to know the appropriate way to compare their similarity. Set A and B are often significantly different in length eg. ...
1
vote
0answers
17 views

Clustering - Distance Metric for Comparing Short Lists of Terms (non-repeating, no frequency)

Clustering involves using some distance or similarity metric. What is the best way to score the similarity of these small sets of words? Criteria: These are technical terms which are extracted from ...
1
vote
0answers
579 views

How do I calculate the Jaccard distance for normal distributions?

I'm trying to replicate the results in this paper. I'm stuck at this part: specifically, calculate Jaccard intersection of pdata1 & pdata2 Is there some neat ...
1
vote
0answers
78 views

Correlation of continuous-time series

I have two continuous time series of discrete events. The data from them are timestamps of occurences, e.g. ...
1
vote
0answers
440 views

Dummy data clustering: OK to apply Euclidian distance hierarchical clustering on Jaccard similarity matrix?

I created dummy variables (binary data) from categorical variables where I want to partition N subjects into multiple classes by some clustering method. I created a Jaccard similarity index matrix for ...
1
vote
0answers
56 views

How to statistically validate that one clustering technique is more effective than other on Binary data

Problem statement - Idea is to cluster students based on their assessment pattern and score. Data = I have a data set that contains Binary data of students answers(Correct/incorrect) which i have ...
1
vote
2answers
394 views

Detecting misclassified data points using distance matrix

I have a data set of around 65,000 items from a public database that have each been assigned to one of around 1,800 classes, but unfortunately I am aware that quite a few are misclassified. My goals ...
1
vote
0answers
159 views

Threshold value for nodes similarity

I am working on an algorithm for community detection on complex systems, where those systems are modeled as graphs. To form community i need to find similarity between nodes. I am using Jaccard ...
1
vote
0answers
241 views

Find similarity score between multiple items based on tags assigned to them

I am working for a client who is in e-commerce. They have thousands of offerings and each has its own set of tags. Now, they want to find similar designs dynamically based on the item selected by the ...
0
votes
0answers
13 views

Can I use the masi distance to compare coders where the text document is not exactly the same?

It seems that Passoneau's MASI distance https://academiccommons.columbia.edu/doi/10.7916/D8PV6TTT was meant to grade different judges on the same document. Could I use this distance method to compare ...
0
votes
0answers
17 views

Is the Bray-Curtis dissimilarity/similarity independent of richness/diversity?

I was told about potential issues with the Bray-Curtis dissimilarity/similarity measure. Specifically, I have been asked if Bray-Curtis is independent of richness/diversity. I am unsure how to answer ...
0
votes
0answers
10 views

Find the hierarchical clustering based on Jaccard Score

I have an excel sheet where I have String details in one column, substring details in one column and jaccard_coefficient in one column respectively :- ...
0
votes
0answers
22 views

How can I get the similarity and dissimilarity of two real numbers?

I would like to calculate the similarity and/or dissimilarity between two real numbers. The challenge is, we can have multiple types of real numbers. Say for example, Case-1: ...
0
votes
0answers
23 views

Choosing a Clustering Method Using Similarity Matrix of Binary Data (in SPSS via Hierarchical: Open to SAS, R, or Others)

I am working on a (somewhat exploratory) clustering of linguistic attributes for data we are working with. In an experiment, we designed prompts to elicit responses, hoping to prompt some sarcasm use. ...
0
votes
0answers
12 views

Measuring similarity between two objects across multiple categories

I'm constructing a crude measure to gauge the similarity between any pairs of objects across multiple dimensions (or categories (for example, they can be percentages of GDP across economic sectors or ...
0
votes
1answer
57 views

Are Jaccard score and F1 score monotonically related?

I have compared the rankings obtained by comparing 10+ classifiers with this two metrics: Jaccard score F1 score They show a perfect correlation. This results holds on 50+ datasets. When comparing ...
0
votes
0answers
12 views

Weighted similarity coefficient for binary data?

I would like to evaluate the accuracy of land degradation maps I have estimated using remote sensing compared against observed field land degradation maps. The maps are classified in a binary way and ...
0
votes
0answers
72 views

What's the advantage of cosine distance over Jaccard distance for text document similarity

We usually use cosine distance as the similarity measure for text document. However, Jaccard distance also somehow make sense to me. My question is for text document, is there any advantage of using ...
0
votes
0answers
124 views

Jaccard/binary (dis)similarity calculation to multidimensional scaling analysis

I have n N x n dataset of features (n) and subjects (N) in which I am attempting to cluster into a lower-dimensional space via multi-dimensional scaling. I'm confused about which MDS setup I need to ...
0
votes
0answers
72 views

Is there a way calculate a gradient of dissimilarity across multiple sites using presence/absence data?

I’ve been trying to come up with a metric for species (dis)similarity for fish in a series based on presence or absence data that gives a value of 1 or 0 for each lake. From my research, it seems like ...
0
votes
1answer
109 views

How to create a “similarity index” on a set of variables

I have a set of 12 variables that describe the dwelling characteristics. 10 of them are dummies (i.e questions like "does the dwelling have a bath?" or "is there crime in the area?"), while one is a 4-...
0
votes
1answer
241 views

Similarity between n >> 2 binary vectors

I have $n$ binary vectors $x_i, ..., x_n$ of length $d$. I wish to compute a similarity measure between all $n$ of them. My initial thought was to use the Jaccard Index, since each binary element $x_{...
0
votes
0answers
261 views

Measure similar to Jaccard index but comparing more than two groups at a time

I am interested in looking at the amount of sharing between several samples (each represented as a vector of character strings), and to do pairwise comparisions, I have been using Jaccard indices. Is ...
0
votes
0answers
239 views

Calculating Similarity between two multiset

I need to find similarity between two multiset. For example, $$A = \{a, a, b, c, d\}$$ $$B = \{a, b, b, d\}$$ I know that I can use Jaccard Index but it gives same weight for each element. As ...
0
votes
0answers
47 views

Similarity between bar plots (counts of categories)?

What are the best ways to compare (calculate similarity) between two bar plots? For example: ...
0
votes
0answers
465 views

Jaccard coefficient (0011 and 1100 most similar?)

I try to compare different binary objects, lets say: ...
0
votes
0answers
294 views

Numerical Similarity Measures for Dirty Data

I need to compare two datasets to see if they are "similar" in terms of numerical similarity between given columns. The problem is, however, that I do not know the characteristics of the 2nd dataset. ...
0
votes
0answers
182 views

Selecting kernel or binary similarity measures

Currently, I am facing a choice of encoding some information either in a binary vector or a normalized (Gaussian) floating point vector of the same length. For instance it could be in the format $[ 1, ...
-1
votes
1answer
123 views

jaccard distance if one samples is a subset

I have a very basic question but I cannot find an answer (especially for a clustering situation). I am trying to do hierarchical clustering of samples using jaccard distances. One sample contains ...