# Should I plan my AB testing accordingly to a minimum number of days or to the number of datapoints?

Here's the thing, I was planning to test a new strategy on my website. A typical AB test for a new homepage. One strategy is the current, the control; the other is the variant, the new strategy. I have a conversion rate of 30%. I'm using a sample size calculator which tells me that my sample size should be 1,335 for each variation. I can get this number of accesses on three days of test. However, I've seen some "AB testing best practices" telling me my test should run for at least a week, due to seasonalities. What should prevail? The number of days (at least one week) or the sample size calculated?

• Before I answer on that question, please tell me what the AB splittests is for, in detail, please tell me about the product, then we could derive if there may be even seasonality Mar 16, 2021 at 21:26
• The sample size obtained from a 'power and sample size' procedure is not a recommendation for the maximum sample size you should use, but a recommendation for the minimum necessary to achieve a certain power. Mar 16, 2021 at 22:15
• @PatrickBormann even though might not be seasonality, let's treat like there is. This is the point to me, to learn how to deal with it, not only on how to solve this specific problem Mar 17, 2021 at 10:38
• @BruceET so, after the minimum number of datapoints in each sample, my results should be more stable, am I right? I mean, if I have 2,000 of datapoints or, 3,000, or 10,000, the result should be the same. Because that's the thing I'm worried about. Mar 17, 2021 at 10:39
• For a correct hypothesis test, yes, you should follow the original data collection and analysis plan (or look into more advanced methods). But if you plan to run your test for a week, irrespective of exact number of points in that week, that is enough. And your calculated sample size is a minimum (for a given power) only. There is no problem in planning for more ... but probably you are following the conversion rate routinely, on a daily basis, and plotting it? Then you will know if there is weakly seasonality! Mar 17, 2021 at 12:13