So I have two groups of products with different sizes and each group has a different group of products in it. I created these groups by ensuring that the difference between these two groups’ cumulative revenues over the last year is as small as possible. I created these groups to use them in A/B test. So I have data of

  1. Cumulative revenues of each group over the last one year (but I can also calculate cumulative weekly/monthly/daily revenues of each group over the last one year).
  2. Yearly individual revenue of each product in each group (again I can calculate weekly/monthly/daily individual revenue of each product in each group).

Now I am looking for such a statistical test that proves or disproves that whichever time period I compare the ratio of these two groups’ total revenues, it falls into some neighbourhood of the current ratio of total revenue of two groups. So that whenever I use these groups in my A/B test I can make it (almost) sure that an increase in ratio beyond my accepted neighbourhood is due to the change of the variable that I want to test its effect.

What might be some feasible ways to do that?

Thank you!

  • $\begingroup$ You’ll have to define what “very close” means, but once you do, this sounds like some kind of equivalence testing. $\endgroup$ – Dave Apr 11 at 14:37
  • $\begingroup$ As I edited above, with "close", I meant no more than what I am currently observing. My current difference is difference for a year though. $\endgroup$ – gülsemin Apr 11 at 14:43
  • $\begingroup$ From your description, you optimized the definition of these groups so that the revenue is as similar as possible. This means your expectation for different time periods is that this difference will tend to be larger. This is a classic problem with A/B testing. $\endgroup$ – zbicyclist Apr 11 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.