0
$\begingroup$

I try to compare different binary objects, lets say:

data <- read.table(header = TRUE, row.names = 1, text = 
"User   va1   var2   var3  var4
abc1     0     0     1     1
abc2     1     1     0     0
abc3     0     0     1     0")

I want to have similar objects in one cluster so I first use the jaccard coefficient to compute similarity:

distance <- dist(data, method = "binary")

Which results in : 

     abc1 abc2 abc3
abc1  0.0      
abc2  1.0   0   
abc3  0.5   1   0.0

Now, what I do not understand here in the first place is: abc1 and abc2 are in my opinion most dissimilar, because they do not have any match. Anyhow, the jaccard coefficient puts them as most similar?.

Now I want to cluster theses guys using hierarchical clustering:

hc <- hclust(distance, method = "ward.D")
plot(hc)

The result again confused me, as abc1 and abc3 are first clustered together before building a bigger cluster with abc2.

So my questions are:
1. Why is jaccard index telling me that abc1 and abc2 are very equal, when they are the most unequal in the dataset?
2. Why is the clustering (based on the distances computed by the jaccard coefficient) then clustering first abc1 and abc3 together (which is right in my opinion) and then merges them with abc2?
3. Do I just need another coefficient?

Improve this question
$\endgroup$
4
  • $\begingroup$ ask only one question per thread $\endgroup$
    – Antoine
    Commented Nov 25, 2016 at 10:02
  • 3
    $\begingroup$ Your matrix of Jaccard coefficients is distance matrix, 1-jaccard. Jaccard is similarity. $\endgroup$
    – ttnphns
    Commented Nov 25, 2016 at 10:03
  • $\begingroup$ Ok, so when I put it right then, abc1 and abc2 are most dissimilar in the distance matrix, which then results in the right clustering $\endgroup$
    – Raphael
    Commented Nov 25, 2016 at 10:15
  • $\begingroup$ ward is only to be used with squared Euclidean distance. jaccard distance and jaccard similarity both exist. Dist=1-sim is their relationship. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Nov 27, 2016 at 21:43

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.