Sample size estimation for test for proportions

Question

I'm interested in finding out if a particular activity increases the chance of a particular segment of prospects converting to a client. I would like to be able to detect a 5% difference in overall conversion rate (i.e., 5% to 10%, or 20% to 25%), and alpha of 5% and some power yet to be determined (80% looks fairly standard, but I think I need to do some cost-benefit analysis). I'm not quite sure how I go about estimating a minimum sample size.

I think that I need to have a control group and treatment group (is this basically an A/B test?) and do a one-tailed z-test of proportions. I have seen some sample size calculators on-line, though they seem to require a baseline conversion rate that I don't think I have. Is it possible to do this without having a baseline conversion rate, and if so, how do I calculate this manually?

Answer 1

In this situation, it is impossible to figure out the sample size required for a specified $\alpha$ and power. However, you can use the conservative method, which is to plug in various values for the unknown baseline conversion rate and see which value gives you the largest required sample size; if you use this for your sample size, then the resulting test (with significance level $\alpha$) should have power greater than or equal to what you specify.