Consider a market basket analysis:
- Among a certain number of transactions two items sometimes occur together. For simplicity let's assume they always occur together, but not in all transactions.
- Now these two items occur together exactly the same number of times and again always together, but among much larger number of transactions. When I apply the common interest measure Lift, the lift in case #2 will be much higher than the lift in case #1, due to the fact that each item individually occurs more rarely.
But this does not feel intuitive. The two items occur more rarely, but their co-occurrence is the same.
An example would be frequent itemsets of Christmas-related items, when looking at grocery store transactions during December compared to a whole year.
I could counteract by somehow incorporating the Confidence. In fact the confidence relationship item#1 => item#2
and vice versa would be 100% for both cases presented above.
Which leads me to my question: Can you point me to any interest measures that incorporates confidence but also accounts for the number of occurences? Or solves in some other way my dilemma described above.
A link to a good paper would be most appreciated but everything, from a good online tutorial to a youtube video, helps.