Jaccard index between set and multiset

Question

Can I use Jaccard index to calculate similarity between set and multiset?

As I know Jaccard is defines as the size of the intersection divided by the size of the union of the sample sets,

that is $J(A, B) = |A \cap B| \, / \, |A \cup B|$

Now if I have a set $s$, $s = \{\text{special}, \text{words}\}$ and a multiset $m$, $m = \{\text{term}, \text{special}, \text{words}, \text{special}\}$

How can I use Jaccard index to take repetition into consideration?

Answer 1

You can use Generalized Jaccard Index, and assume that the set $s$ is actually a multiset:

If $\mathbf{x} = (x_1, x_2, \ldots, x_n)$ and $\mathbf{y} = (y_1, y_2, \ldots, y_n)$ are two vectors with all real $x_i, y_i \geq 0$, then their Jaccard similarity coefficient is defined as $$J(\mathbf{x}, \mathbf{y}) = \frac{\sum_i \min(x_i, y_i)}{\sum_i \max(x_i, y_i)}.$$

Here you can read "vector" as "multiset", and $x_i$ is a count of element $i$ in the multiset $\mathbf x$.

Answer 2

How do you want to take the repetition into account? I sew a few ways you could do it:

1. You can just ignore it, the resulting index value would be $J(s,m)=2/3$ ;
2. You can count the repeated words as if they were different words. Then it would be $J(s,m)=2/4$, now the union is 4 because 'special' appears twice in the union;
3. You can count the repeated words with a different weight. For instance 0.5, $J(s,m)=2/3.5$, 'special' gets a weight of 1.5, 1 for the first appearance and 0.5 for the second.

But I would say it depends on what you are trying to do and how you are approaching your problem.