I'm hoping that people will indulge me a beginner question, but I was recently learning about association rule learning from a lecture on Information Retrieval. (For reference, the video I was watching is here). As a disclaimer, I'm still new to this area, so hopefully I'm not missing something obvious.

So, my understanding based on the video and the Wikipedia article is that you have some rule about a certain set of conditions that "makes" some other condition likely, written in the form of $\{i_1, i_2, ... i_n\} \rightarrow j$. For example, if you buy ketchup and mustard at the store, it's likely that you'll also buy hamburger meat. Furthermore, each rule has support, which is the frequency with which it appears in your data set.

So, why not just formulate this as conditional probability? In fact, the confidence in a rule is formulated in terms of conditional probability. For example, rather than having an association rule $\{milk, bread\} \implies \{beer\}$, why not just say that the probability of the customer purchasing beer given that they purchased milk and bread is $x$? What does this concept actually "buy" you beyond what you could get by simply using conditional probability? Or is it more about having a method of discovering the relationship in the first place?

Also (and please let me know if this should be a separate question), how do you apply this without being biased against items that are unusual to begin with? For example, suppose that it's very unusual to buy caviar and it's even more unusual to buy both caviar and anchovies, but when people do buy them together there's a 90% chance of also buying chili powder (and I'm just throwing things out here). If I'm understanding this correctly, that would have low support because it wouldn't appear very often in your data set. That being the case, how could you ever have association rules about stuff like caviar?

Or do we just say that, in this case, the sample size may be too small to make a meaningful inference (i.e. we can't really say for sure if caviar and anchovies actually means that you're more likely to buy chili powder - we're not sure if this is a real relationship)?

I suppose that another possibility is that people don't particularly care about the fact that people who buy anchovies and caviar also buy chili powder, it's not like people are exactly buying anchovies and caviar every day.

A more concrete example: I recently purchased a book called Behavioral Operational Research: Theory, Methodology and Practice on Amazon.com. It's not a very commonly-purchased book (#538,076 in books, and I was the first reviewer). In a fit of cheapness, I also purchased an old edition of Operations Research: An Introduction (#2,738,505 in books). Finally, I purchased Cognitive Modeling (MIT Press, #2,738,505 in books) at around the same time as I got the others and for similar reasons (all three were as part of investigation into possible topics for upcoming research). However, all three are rare. You could argue one of two ways here: either that buying Behavioral Operational Research really does make you more likely to buy slightly old editions of Operations Research: An Introduction, or you could argue that I'm too small of a sample to tell if there's actually any relationship.

TL;DR Why do we need association rules when we can just use conditional probability instead? And how do you use the idea of association rules to generate predictions about items that are rare to begin with?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.