I have data that is equivalent to:

shopper_1 = ['beer', 'eggs', 'water',...]
shopper_2 = ['diapers', 'beer',...]

I would like to do some analysis on this data set to get a correlation matrix that would have an implication similar to: if you bought x, you are likely to buy y.

Using python (or perhaps anything but MATLAB), how can I go about that? Some basic guidelines, or pointers to where I should look would help.

Thank you,

Edit - What I have learned:

  1. These kinds of problems are known as association rule discovery. Wikipedia has a good article covering some of the common algorithms to do so. The classic algorithm to do so seems to be Apriori, due Agrawal et. al.

  2. That lead me to orange, a python interfaced data mining package. For Linux, the best way to install it seems to be from source using the supplied setup.py

  3. Orange by default reads input from files, formatted in one of several supported ways.

  4. Finally, a simple Apriori association rule learning is simple in orange.

  • 3
    $\begingroup$ If you were looking for an R package, arules would be worth a look. Maybe "association rules" is a good search term $\endgroup$ – Karsten W. Mar 8 '11 at 13:06
  • 2
    $\begingroup$ See also the Apriori algorithm for the "standard" approach to this problem. $\endgroup$ – cardinal Mar 8 '11 at 16:53

In addition to the links that were given in comments, here are some further pointers:

About Python, I guess now you have an idea of what you should be looking for, but the Orange data mining package features a package on Association rules and Itemsets (although for the latter I cannot found any reference on the website).


I recently came across pysuggest which is

a Top-N recommendation engine that implements a variety of recommendation algorithms. Top-N recommender systems, a personalized information filtering technology, are used to identify a set of N items that will be of interest to a certain user. In recent years, top-N recommender systems have been used in a number of different applications such to recommend products a customer will most likely buy; recommend movies, TV programs, or music a user will find enjoyable; identify web-pages that will be of interest; or even suggest alternate ways of searching for information.

  • $\begingroup$ How many products, I wonder, need to be involved before a simple correlation matrix is insufficient? $\endgroup$ – rolando2 Apr 11 '11 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.