6
$\begingroup$

I want to compare 2 vectors of length 43; they have values of 0 (not present) and 1 (present). I will refer to $M_{1,1}$ as situations in which both 1 are present, and $M_{1,0}$ and $M_{0,1}$ to situations in with only one 1 is present while the other value is 0.

data3$IDS  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 
       0 0 0 0 0 0 0 0 0 0
data3$CESD 1 1 1 0 1 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1
           1 1 1 1 1 1 1 1 1 1 

I want to understand how related these 2 vectors are. Reading up on the topic, the Jaccard index seems the way to go. In this specific case, the Jaccard index would be (note that I am using the formula given next to the second figure on Wikipedia): $$ \frac{M_{1,1}}{(M_{1,0} + M_{0,1} - M_{1,1})} $$ In my case: $8 / (23 + 12 - 8) = 0.2962963$

Using:

library('clusteval')
cluster_similarity(data3$IDS, data3$CESD, similarity="jaccard", method="independence")

Returns:

0.553429

I can't quite figure out why, and where the mistake is that I make.

Another thing I do not understand is in cases of high overlap. Imagine $M_{1,1} = 30$, with only $2$ values each in the cells $M_{1,0}$ and $M_{0,1}$. This would lead to a Jaccard index of $30/(2+2-30) = -1.153846$.

But the J index is only defined between 0 and 1. Where is my misunderstanding?

$\endgroup$
10
$\begingroup$

Looking at the Wikipedia page's edit history, it seems the problem was due to a confusion about the two types of mathematical notation that are used to represent the index. Using notation from set theory, we have:
$$ J(A,B) = \frac{|A\cap B|}{|A\cup B|} = \frac{|A\cap B|}{|A| + |B| - |A\cap B|} $$ where $\cap$ denotes the intersection, $\cup$ denotes the union, and $\lvert\ \rvert$ denotes the cardinality.

Lower down, the formula was presented algebraically using counts from a matrix / contingency table $M$:
$$ J = \frac{M_{11}}{M_{10}+M_{01}+M_{11}} $$ This seemed contradictory to an editor who commented that there was an "Erro in formula [sic]. Should be minus the intersection".

The two formulas are in fact consistent because although $|A\cap B|=M_{11}$, $|A|\ne M_{10}$ and $|B|\ne M_{01}$. The algebraic formula could have been presented (in a manner that is more cumbersome, but more clearly parallel to the top formula) like this:
$$ J = \frac{M_{11}}{\sum_j M_{1j} + \sum_i M_{i1} - M_{11}} $$

$\endgroup$
9
$\begingroup$

That formula is wrong indeed.

It should be m11 / (m01 + m10 + m11), since the Jaccard index is the size of the intersection between two sets, divided by the size of the union between those sets.

The correct value is 8 / (12 + 23 + 8) = 0.186. I find it weird though, that this is not the same value you get from the R package.

You understood correctly that the Jaccard index is a value between 0 and 1. For the example you gave the correct index is 30 / (2 + 2 + 30) = 0.882.

$\endgroup$
  • $\begingroup$ The package has issues with 0 and 1, and estimates the wrong contingency table. I have not been able to find the reason for it as of yet, so I wrote my own J function with the corrected formula. The dist.binary function from the package 'ade4' also works like a charm (in this case, use 1-J). $\endgroup$ – Torvon Oct 13 '15 at 0:34
  • $\begingroup$ The clusteval package is meant for comparing two clusterings (partitions) rather than two sets. Jaccard Index for comparing sets assumes 0=absence from set and 1=presence in set. If you instead interpret the 0/1 as indicating the two possible cluster labels, with data3$IDS and data3$CESD as separate clusterings, then clusteval gives the correct answer. A quick way to check is that cluster labels are invariant to permutation, whereas presence/absence is not. So, reversing the 0/1 in one of the variables changes the set-based Jaccard but not the clustering/ clusteval Jaccard. $\endgroup$ – Will Townes Aug 22 '18 at 15:23
5
$\begingroup$

I wrote a simple function for calculating the Jaccard index (similarity coefficient) and the complementary Jaccard distance for binary attributes:

# Your dataset
df2 <- data.frame(
  IDS = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), 
  CESD = c(1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1))

# Function returns the Jaccard index and Jaccard distance
jaccard <- function(df, margin) {
  if (margin == 1 | margin == 2) {
    M_00 <- apply(df, margin, sum) == 0
    M_11 <- apply(df, margin, sum) == 2
    if (margin == 1) {
      df <- df[!M_00, ]
      JSim <- sum(M_11) / nrow(df)
    } else {
      df <- df[, !M_00]
      JSim <- sum(M_11) / length(df)
    }
    JDist <- 1 - JSim
    return(c(JSim = JSim, JDist = JDist))
  } else break
}

The function takes two arguments: x a dataframe or matrix object, and m the MARGIN argument used in the apply function. If your data is in wide format set m = 2 to apply sum over the columns. If your data is in long format set m = 1 to apply sum over the rows.

> jaccard(df2, 1)
     JSim     JDist 
0.1860465 0.8139535 
$\endgroup$
1
$\begingroup$

Solved. The problem was that Wikipedia actually is wrong, specifically the formula:

m11/(m10+m01-m11)

$\endgroup$
1
$\begingroup$

Augmentation of @jsb's code to enable a similarity measure across all observations.

# Jaccar Index
library(dplyr)

# Your dataset
df <- data.frame(t(data.frame(c1=rnorm(100),
                              c2=rnorm(100),
                              c3=rnorm(100),
                              c4=rnorm(100),
                              c5=rnorm(100),
                              c6=rnorm(100))))

df[df > 0] <- 1
df[df <= 0] <- 0
df

# Function returns the Jaccard index and Jaccard distance
# Parameters:
# 1. df, dataframe of interest
# 2. margin, axis in which the apply function is meant to move along
jaccard <- function(df, margin=1) {
  if (margin == 1 | margin == 2) {
    M_00 <- apply(df, margin, sum) == 0
    M_11 <- apply(df, margin, sum) == 2
    if (margin == 1) {
      df <- df[!M_00, ]
      JSim <- sum(M_11) / nrow(df)
    } else {
      df <- df[, !M_00]
      JSim <- sum(M_11) / length(df)
    }
    JDist <- 1 - JSim
    return(c(JSim = JSim, JDist = JDist))
  } else break
}

jaccard(df[1:2,], margin=2)


jaccard_per_row <- function(df, margin=1){
   require(magrittr)
   require(dplyr)
   key_pairs <- expand.grid(row.names(df), row.names(df))
   results <- t(apply(key_pairs, 1, function(row) jaccard(df[c(row[1], row[2]),], margin=margin)))
   key_pair <- key_pairs %>% mutate(pair = paste(Var1,"_",Var2,sep=""))
   results <- data.frame(results)
   row.names(results) <- key_pair$pair
   results
}

jaccard_per_row(df, margin=2)

Output:

           JSim     JDist
c1_c1 1.0000000 0.0000000
c2_c1 0.3974359 0.6025641
c3_c1 0.3513514 0.6486486
c4_c1 0.3466667 0.6533333
c5_c1 0.3333333 0.6666667
c6_c1 0.3888889 0.6111111
c1_c2 0.3974359 0.6025641
c2_c2 1.0000000 0.0000000
c3_c2 0.3289474 0.6710526
c4_c2 0.4166667 0.5833333
c5_c2 0.3466667 0.6533333
c6_c2 0.3289474 0.6710526
c1_c3 0.3513514 0.6486486
c2_c3 0.3289474 0.6710526
c3_c3 1.0000000 0.0000000
c4_c3 0.2236842 0.7763158
c5_c3 0.3333333 0.6666667
c6_c3 0.3529412 0.6470588
c1_c4 0.3466667 0.6533333
c2_c4 0.4166667 0.5833333
c3_c4 0.2236842 0.7763158
c4_c4 1.0000000 0.0000000
c5_c4 0.3676471 0.6323529
c6_c4 0.2236842 0.7763158
c1_c5 0.3333333 0.6666667
c2_c5 0.3466667 0.6533333
c3_c5 0.3333333 0.6666667
c4_c5 0.3676471 0.6323529
c5_c5 1.0000000 0.0000000
c6_c5 0.2957746 0.7042254
c1_c6 0.3888889 0.6111111
c2_c6 0.3289474 0.6710526
c3_c6 0.3529412 0.6470588
c4_c6 0.2236842 0.7763158
c5_c6 0.2957746 0.7042254
c6_c6 1.0000000 0.0000000

Now you can determine which rows have a threshold above a desired percentage, and only keep very similar observations for your analysis.

Enjoy!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.