I have a data for 322 items that are purchased between 2014 and 2018 by men and women. If I ran a Wilcoxon Signed-Rank test with a Benjamini-Hochberg correction on the items purchased by men and women over the five year purpose I find that 83% are significant (<= 0.05) and 17% are not significant (>0.05).

But if I then calculate the average purchases of men and women over the five year and run the test again, I find no significance whatsoever. I'm not sure which test I should use. Shouldn't the average also show significance?

One is a test over each item over the five years and the other is an average over all items in each year.


In aggregate, there is neither reason nor guarantee that you will observe the same trend as when you split the comparison by some variable (product in your case). This phenomenon is called Simpson's paradox.

Since you have only mentioned significance, and not the direction of the effects, it could also simply be the case that the trends observed within products cancel each other out when averaged.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.