1
$\begingroup$

I am aware the Jaccard similarity coefficient, $J$, is classically defined as the cardinality of the intersection divided by the cardinality of the union of two sets, $A$ and $B$. I can easily understand and comprehend this equation.

$$J=\frac{|A\cap B|}{|A \cup B|}=\frac{|A\cap B|}{|A|+|B|-|A\cap B|}$$

However, I have also come across another expression for the Jaccard similarity coefficient using vector data, $p$ and $q$, instead of sets (see, for example, page 302, Equation 22 in Cha 2007)

$$J=\frac{\sum_{i=1}^n p_i q_i}{\sum_{i=1}^n p_i^2 + \sum_{i=1}^n q_i^2 -\sum_{i=1}^n p_i q_i}$$

This expression is not as obvious to me in terms of assessing the similarity between the vectors. I have been trying to find a reference that shows how expression for vector data is derived and how that relates to the set theoretic expression of the Jaccard similarity coefficient - but without any luck. Any help or suggestions welcome.

Reference: Cha, S-H. (2007). Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions. Journal of Mathematical Models and Methods in Applied Sciences, 4:300-307.

$\endgroup$

1 Answer 1

2
$\begingroup$

Presumably the vectors contain zeros or ones and refer to two sets being compared.

$p_i$ or $q_i$ is $1$ if element $i$ is in one set or another and $0$ otherwise.

Then $\sum p_i^2$ is the total number of elements in the first set (and incidentally it is equal to $\sum p_i$ as $1^2 = 1$). $\sum q_i^2$ is to be interpreted in the same way for the second set.

Each $p_i q_i$ is $1$ if and only if both $p_i$ and $q_i$ are $1$. Otherwise it is $0$, for all three cases $p_i = 0, q_i = 0; p_i = 1, q_i = 0; p_i = 0, q_i = 1$. So $\sum p_i q_i$ counts the number of elements in the intersection of the two sets.

I don't have a reference and don't recall seeing Jaccard defined in this way, but it would be natural enough as a step towards code.

$\endgroup$
2
  • $\begingroup$ Thanks for the answer. I have added a reference to the question (there are many more I have). Also I am not sure the vectors are limited to zeros and ones as this applies to probability density functions, hence they would be unit vectors (as implied in reference). $\endgroup$
    – anna6931
    Nov 14, 2023 at 2:09
  • 1
    $\begingroup$ Thanks for the reference. Sorry, but I don't now have the time or inclination to study it closely. But clearly if $p$ can have values other than $0$ or $1$, then it's no longer true that (e.g.) $\sum p_i = \sum p_i^2$. $\endgroup$
    – Nick Cox
    Nov 14, 2023 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.