It is well-known that it is problematic to stop an A/B test before we obtain a required number of samples. This website "A/B Testing Rigorously (without losing your job)" explains how to stop safely an A/B test in the middle of the test.

At the website we define maximum cumulative decision limit by


Here $p$ is the significance level and $n$ is the number of "conversion".

My question is: where does the formula come from?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.