3
$\begingroup$

I need to cluster products based on market basket data, i.e. I have a data table with sold products and the respective orders and I want to cluster products so that products within a group are bought together quite frequently.

To my knowledge this is a task which relates to two topics, on the one hand association rule learning and on the other hand clustering. What I have done so far is to compute pairwise similarities based on the Jaccard coefficient and use a PAM approch to cluster the products.

Unfortunately the results are very poor in matters of the quality of the clustering solution. My question is therefore if the general approach seem plausible and only the results are poor or if I should use another approach? I would appreciate any idea.

Kind Regards,
Daniel

$\endgroup$
1
$\begingroup$

You can consider frequent itemsets to be a specific form of clustering designed for market basket data. On such data, it is much more meaningful than what you would get with a traditional partitioning algorithm like k-means. K-means needs to put every item into a cluster, and you need to know the number of clusters beforehand. Frequent itemset mining can handle that you may have items that are barely ever (or never) sold, and for which you do not have enough data to assign them in any meaningful way. That is why you use frequent itemset mining and not clustering.

$\endgroup$
  • $\begingroup$ Actually frequent itemsets are what the similarities are based upon in my approach. But if I want a partition of my products into groups two problems occur if I stop after extracting the frequent itemsets. First, there are frequent itemsets that include the same product. This is not a roblem itself but it does not mean that the other products of these itemsets are sold together frequently. Second, there are products that are not included in any frequent itemset at all. $\endgroup$ – bratwoorst711 Feb 29 '16 at 12:41
  • $\begingroup$ That is what I'm saying. Some items are never or barely ever sold. There is no good cluster assignemnt for them, so you better don't assign them to clusters at all! $\endgroup$ – Anony-Mousse Feb 29 '16 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.