Speed up web a/b tests with sample size checkpoints Before starting an a/b test with a control and one experimental route, I can calculate a required sample size based on conversion rate estimates for both routes. I can get a good estimate of the conversion rate of the control by looking at historical data. But the conversion rate of the experimental route is unknown. What I would like to do is calculate a number of different sample sizes based on a variety of sensitivities.
For example, I can calculate sample sizes for a 10%, 15% and 20% sensitivity (increase in conversion from the control) that might look like this:
Sensitivity   Required Sample Size
10%           1,961
15%           871
20%           490

Some of the reading I've done says that you should calculate a single sample size at the start of the test and always run the test for that long. 
Question:


*

*Is there any problem with checking for statistical significance at multiple pre-calculated sample sizes and potentially ending a test early if I've found the results to be statistically significant? 


Example:
I originally estimate that the experimental route will outperform the control by 15%. But once I've reached 490 samples I find that the experimental route is actually outperforming the control by 20%, can I end the test and declare that the experimental route boosts conversion by 20%?
 A: This approach doesn't have the properties you would have if you fixed the sample size ahead of time. 
The situation where you look for a particular result while your experiment continues and have some 'stopping rule' (halt your experiment early if a particular situation is achieved) is a version of sequential analysis; see also SPRT.
You have to take care that the properties of your actual decision rules are doing what you want - you can't apply the properties of one situation to another and expect that it will work.
For example, you won't have the power you have calculated at the given sample sizes if you're doing sequential testing; the required sample sizes will be somewhat larger. On the other hand, when your effects are substantial, you'll often end up stopping earlier - meaning smaller sample sizes/faster decisions.

Specifically which properties are affected if one were to terminate the test at, say, 490 samples because 20% improvement over the control is shown? 

First, estimates will be biased, but also standard errors, Type I and (as already mentioned) Type II error rates are affected - plus anything any of these will impact. 
The SPRT link I gave outlines a general approach that is used with early stopping with hypothesis testing.

Phillip Good does some work with discrete sequential analysis in his book
Permutation, Parametric, and Bootstrap Tests of Hypotheses in section 6.7
