I am using Apriori to find association rules, where each instance is a vector of True/False value, indicating whether an item is bought or not. Suppose that we obtain a rule b=>a with confidence c, which means that P(a|b)=c. Then, we can compute the lift of the rule by P(a|b)/P(a).

Let P(A) denote the distribution where P(A=True)=P(a), and P(A=False)=1-P(a). I think if P(A|b) and P(A) are not statistically different, we might not say there is a real lift. It happens because my data set is small. So I want to run a significance test between P(A) and P(A|b). As we can see, the data for P(A) is from the entire data set, and P(A|b) is from the subset of data for P(A).

I am wondering which test I should use. I used Welch's t-test, but I am not sure if it is ok for me, and whether I should use other tests instead. Thank you for your advice!


I think your statement that P(A|b) is from a subset is incorrect. If b then A but also if not b then what? The second situation is also part of the condition. In this case you can construct two likelihood functions and compare them. The first is without knowing b, the second is with knowing it. That is the classical way to compare to regression models. It should work here.

| cite | improve this answer | |
  • $\begingroup$ For example, suppose that we have 3 instances: (A, B) = [(T, T), (T, F), (F, T)] From above, we have P(A=T) = 0.67, and P(A=T|b) = 0.5. I really used part of the data for P(A) to compute P(A|b). And, I think I have to compare P(A|b) with P(A), not with P(A| not b), because the association rule uses P(A|b)/P(A) as lift. Or, should I compare P(A|b) and P(A| not b) to guarantee that data are not overlapping? $\endgroup$ – Efu May 20 '18 at 3:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.