A quick question on the frequentest approach of sample size calculation.
I recently came across this below note:
The sample size required should be strictly enforced; if the power analysis shows that each test variant requires 100,000 users, the test should only be evaluated based on the first 100,000 users that are bucketed. If this is not strictly adhered to, we expose ourselves to uncertainty
I was under the assumption that the sample size calculation provides the minimum sample size that is required to measure the difference between the treatment and control and is related to the minimum detectable effect but having more data would not necessarily be a problem.