Here's the thing, I was planning to test a new strategy on my website. A typical AB test for a new homepage. One strategy is the current, the control; the other is the variant, the new strategy. I have a conversion rate of 30%. I'm using a sample size calculator which tells me that my sample size should be 1,335 for each variation. I can get this number of accesses on three days of test. However, I've seen some "AB testing best practices" telling me my test should run for at least a week, due to seasonalities. What should prevail? The number of days (at least one week) or the sample size calculated?
The sample size of the ab test is based on the number of data points, or randomization units, not the number of experiment days.
The recommendation that running experiments for a minimum of one week is about external validity reasons: you may have a different population of users on weekends or weekdays, even the same user may behave differently. If you only run an experiment on weekends, you can't directly generalize experimental effects to weekdays.
Reference: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing, chapter 2