# calculating necessary sample size

I am trying to use python and online tools to calculate the accurate sample size. However, each way I use I get a different result.

This is the data I have from a previous test

Control Group

• Sent:140000
• Converted: 6000
• Conversion Rate: 0.0429

Test/Treatment:

• Sent:350000
• Converted:19000
• Conversion Rate: 0.0543

If I use this calculator http://www.evanmiller.org/ab-testing/sample-size.html with

• Baseline conversion rate 4.29
• minimum detectable effect 2% ( I don't get results for 20%)
• 1 - Beta: 80%
• alpha: 5%
• RESULT: 1,713

If I use this calculator https://www.optimizely.com/resources/sample-size-calculator/?conversion=4.29&effect=20&significance=95 with

• Baseline conversion rate 4.29
• minimum detectable effect: 20%
• statistical significance 95%
• RESULT: 8,300

same calculator

• Baseline conversion rate 4.29
• minimum detectable effect 20% (if I enter 2%I get a sample size required of 1,200,000)
• statistical significance 90%
• RESULT: 7,900

Finally, when I use python (code from here https://stackoverflow.com/questions/15204070/is-there-a-python-scipy-function-to-determine-parameters-needed-to-obtain-a-ta)

from scipy.stats import norm, zscore

def sample_power_probtest(p1, p2, power=0.8, sig=0.05):
z = norm.isf([sig/2]) #two-sided t test
zp = -1 * norm.isf([power])
d = (p1-p2)
s =2*((p1+p2) /2)*(1-((p1+p2) /2))
n = s * ((zp + z)**2) / (d**2)
return int(round(n[0]))

def sample_power_difftest(d, s, power=0.8, sig=0.05):
z = norm.isf([sig/2])
zp = -1 * norm.isf([power])
n = s * ((zp + z)**2) / (d**2)
return int(round(n[0]))

if __name__ == '__main__':
n = sample_power_probtest(0.0429, 0.0543, power=0.8, sig=0.05)
print n


and I get RESULT: 5585

You need to enter a 20% relative effect into the first calculator. The result is 9,000 samples. The actual effect is 26% that might be the reason your program returns a different result.

• Welcome to the site! This is more of a comment than an answer, however - we favour in depth answers. This page offers advice on how to answer: stats.stackexchange.com/help/how-to-answer – mkt - Reinstate Monica Mar 31 '18 at 0:32
• Sorry, but my currently low reputation prevents me from leaving comments for the original question :) – iggy Apr 1 '18 at 5:27

This is an old question but it may be useful to add an answer.

Using your values on this reputed online sample size calculator: http://www.sample-size.net/sample-size-proportions/

The standard normal deviate for α = Zα = 1.960
The standard normal deviate for β = Zβ = 0.842
Pooled proportion = P = (q1*P1) + (q0*P0) = 0.049
A = Zα√P(1-P)(1/q1 + 1/q0) = 0.843
B = Zβ√P1(1-P1)(1/q1) + P0(1-P0)(1/q0) = 0.362
C = (P1-P0)2 = 0.000
Total group size = N = (A+B)2/C = 11,168
Continuity correction (added to N for Group 0) = CC = 1/(q1 * |P1-P0|) = 175


Hence 5584 per group.