# Apriori versus Jaccard Coeff for recommender-system

I developed a market basket program for a retailer. I used the pymining library association rules function. I assume this uses the Apriori algorithm. I got 340,000 rules for an input of 17,000 invoices each with a list of corresponding items.

My colleague developed a R program using Jaccard coefficient for the same set of 17,000 invoice items. The R program gave a set of 9000 similarity coefficients for combinations of 2 items. The store has 350 active items. This algorithm assumes that if 2 items were bought in the same basket, they were 'similar'. This had to be done because there is no rating available in this store.

I then displayed both our results in a common grid. The columns were Items in basket, items recommended by pymining, support, confidence, items recommended by Jaccard coefficient.

So for example, if I had items [1, 2, 3] in the basket, and [4, 5] in pymining recommended, I picked the items with highest similarity for [1,2,3] from the R output, and got say [4,7,8].

What I find is that there is very little correspondence between the recommendations of the market basket and the Jaccard coefficient.

• Is this expected behavior?
• Can we expect the outputs of these 2 algorithms to give similar outputs?
• Do you have a reference or at least a name for the algorithm used by pymining? – shadowtalker Nov 25 '15 at 22:26
• @ssdecontrol The author of pymining has referenced cs.uic.edu/~liub/teach/cs583-fall-05/… . This PPT talks of Apriori so i assume pymining uses Apriori. – Chakra Nov 26 '15 at 3:46

This Apriori frequent itemsets algorithm gives different results because it's looking for something different. The Jaccard index is a unitless, relative index. An association rule, to quote the presentation, "is a pattern that states when $X$ occurs, $Y$ occurs with certain probability." But $X$ and $Y$ aren't just individual items: they're subsets of the itemset, meaning that association rules can involve more than two items. Moreover, an association rule is a specific kind of itemset that minimally satisfies particular criteria for minimum probability of occurrence, including two that are set as parameters for running the algorithm.