Jaccard similarity in R

Question

I want to compare 2 vectors of length 43; they have values of 0 (not present) and 1 (present). I will refer to $M_{1,1}$ as situations in which both 1 are present, and $M_{1,0}$ and $M_{0,1}$ to situations in with only one 1 is present while the other value is 0.

data3$IDS  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 
       0 0 0 0 0 0 0 0 0 0
data3$CESD 1 1 1 0 1 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1
           1 1 1 1 1 1 1 1 1 1 

I want to understand how related these 2 vectors are. Reading up on the topic, the Jaccard index seems the way to go. In this specific case, the Jaccard index would be (note that I am using the formula given next to the second figure on Wikipedia): $$ \frac{M_{1,1}}{(M_{1,0} + M_{0,1} - M_{1,1})} $$ In my case: $8 / (23 + 12 - 8) = 0.2962963$

Using:

library('clusteval')
cluster_similarity(data3$IDS, data3$CESD, similarity="jaccard", method="independence")

Returns:

0.553429

I can't quite figure out why, and where the mistake is that I make.

Another thing I do not understand is in cases of high overlap. Imagine $M_{1,1} = 30$, with only $2$ values each in the cells $M_{1,0}$ and $M_{0,1}$. This would lead to a Jaccard index of $30/(2+2-30) = -1.153846$.

But the J index is only defined between 0 and 1. Where is my misunderstanding?

Answer 1

Looking at the Wikipedia page's edit history, it seems the problem was due to a confusion about the two types of mathematical notation that are used to represent the index. Using notation from set theory, we have:
$$ J(A,B) = \frac{|A\cap B|}{|A\cup B|} = \frac{|A\cap B|}{|A| + |B| - |A\cap B|} $$ where $\cap$ denotes the intersection, $\cup$ denotes the union, and $\lvert\ \rvert$ denotes the cardinality.

Lower down, the formula was presented algebraically using counts from a matrix / contingency table $M$:
$$ J = \frac{M_{11}}{M_{10}+M_{01}+M_{11}} $$ This seemed contradictory to an editor who commented that there was an "Erro in formula [sic]. Should be minus the intersection".

The two formulas are in fact consistent because although $|A\cap B|=M_{11}$, $|A|\ne M_{10}$ and $|B|\ne M_{01}$. The algebraic formula could have been presented (in a manner that is more cumbersome, but more clearly parallel to the top formula) like this:
$$ J = \frac{M_{11}}{\sum_j M_{1j} + \sum_i M_{i1} - M_{11}} $$

Answer 2

That formula is wrong indeed.

It should be m11 / (m01 + m10 + m11), since the Jaccard index is the size of the intersection between two sets, divided by the size of the union between those sets.

The correct value is 8 / (12 + 23 + 8) = 0.186. I find it weird though, that this is not the same value you get from the R package.

You understood correctly that the Jaccard index is a value between 0 and 1. For the example you gave the correct index is 30 / (2 + 2 + 30) = 0.882.

Answer 3

I wrote a simple function for calculating the Jaccard index (similarity coefficient) and the complementary Jaccard distance for binary attributes:

# Your dataset
df2 <- data.frame(
  IDS = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), 
  CESD = c(1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1))

# Function returns the Jaccard index and Jaccard distance
jaccard <- function(df, margin) {
  if (margin == 1 | margin == 2) {
    M_00 <- apply(df, margin, sum) == 0
    M_11 <- apply(df, margin, sum) == 2
    if (margin == 1) {
      df <- df[!M_00, ]
      JSim <- sum(M_11) / nrow(df)
    } else {
      df <- df[, !M_00]
      JSim <- sum(M_11) / length(df)
    }
    JDist <- 1 - JSim
    return(c(JSim = JSim, JDist = JDist))
  } else break
}

The function takes two arguments: x a dataframe or matrix object, and m the MARGIN argument used in the apply function. If your data is in wide format set m = 2 to apply sum over the columns. If your data is in long format set m = 1 to apply sum over the rows.

> jaccard(df2, 1)
     JSim     JDist 
0.1860465 0.8139535 

Answer 4

Solved. The problem was that Wikipedia actually is wrong, specifically the formula:

m11/(m10+m01-m11)

Answer 5

Augmentation of @jsb's code to enable a similarity measure across all observations.

# Jaccar Index
library(dplyr)

# Your dataset
df <- data.frame(t(data.frame(c1=rnorm(100),
                              c2=rnorm(100),
                              c3=rnorm(100),
                              c4=rnorm(100),
                              c5=rnorm(100),
                              c6=rnorm(100))))

df[df > 0] <- 1
df[df <= 0] <- 0
df

# Function returns the Jaccard index and Jaccard distance
# Parameters:
# 1. df, dataframe of interest
# 2. margin, axis in which the apply function is meant to move along
jaccard <- function(df, margin=1) {
  if (margin == 1 | margin == 2) {
    M_00 <- apply(df, margin, sum) == 0
    M_11 <- apply(df, margin, sum) == 2
    if (margin == 1) {
      df <- df[!M_00, ]
      JSim <- sum(M_11) / nrow(df)
    } else {
      df <- df[, !M_00]
      JSim <- sum(M_11) / length(df)
    }
    JDist <- 1 - JSim
    return(c(JSim = JSim, JDist = JDist))
  } else break
}

jaccard(df[1:2,], margin=2)


jaccard_per_row <- function(df, margin=1){
   require(magrittr)
   require(dplyr)
   key_pairs <- expand.grid(row.names(df), row.names(df))
   results <- t(apply(key_pairs, 1, function(row) jaccard(df[c(row[1], row[2]),], margin=margin)))
   key_pair <- key_pairs %>% mutate(pair = paste(Var1,"_",Var2,sep=""))
   results <- data.frame(results)
   row.names(results) <- key_pair$pair
   results
}

jaccard_per_row(df, margin=2)

Output:

           JSim     JDist
c1_c1 1.0000000 0.0000000
c2_c1 0.3974359 0.6025641
c3_c1 0.3513514 0.6486486
c4_c1 0.3466667 0.6533333
c5_c1 0.3333333 0.6666667
c6_c1 0.3888889 0.6111111
c1_c2 0.3974359 0.6025641
c2_c2 1.0000000 0.0000000
c3_c2 0.3289474 0.6710526
c4_c2 0.4166667 0.5833333
c5_c2 0.3466667 0.6533333
c6_c2 0.3289474 0.6710526
c1_c3 0.3513514 0.6486486
c2_c3 0.3289474 0.6710526
c3_c3 1.0000000 0.0000000
c4_c3 0.2236842 0.7763158
c5_c3 0.3333333 0.6666667
c6_c3 0.3529412 0.6470588
c1_c4 0.3466667 0.6533333
c2_c4 0.4166667 0.5833333
c3_c4 0.2236842 0.7763158
c4_c4 1.0000000 0.0000000
c5_c4 0.3676471 0.6323529
c6_c4 0.2236842 0.7763158
c1_c5 0.3333333 0.6666667
c2_c5 0.3466667 0.6533333
c3_c5 0.3333333 0.6666667
c4_c5 0.3676471 0.6323529
c5_c5 1.0000000 0.0000000
c6_c5 0.2957746 0.7042254
c1_c6 0.3888889 0.6111111
c2_c6 0.3289474 0.6710526
c3_c6 0.3529412 0.6470588
c4_c6 0.2236842 0.7763158
c5_c6 0.2957746 0.7042254
c6_c6 1.0000000 0.0000000

Now you can determine which rows have a threshold above a desired percentage, and only keep very similar observations for your analysis.

Enjoy!