In various statistical software programs (and, allegedly, in some online 'calculators')
you can specify typical proportions that you'd like to be able to distinguish at the 5% level of significance and with power 80%.
Specifically, if reasonable proportions for Treatments 1 and 2 are $p_1 = 0.5$ and $p_2 = 0.6,$ then these are the 'proportions' you enter. (Of course, you won't know the exact proportions, but the difference between them should be the
size of difference you'd like to be able to detect.)
Sample size computation from Minitab. In particular, output from a 'power and sample size' procedure in a
recent release of Minitab is shown below. For a two-sided test with the proportions guessed above, you'd need $n=388$ in each
group for 80% power.
Power and Sample Size
Test for Two Proportions
Testing comparison p = baseline p (versus ≠)
Calculating power for baseline p = 0.5
α = 0.05
Comparison p Size Power Actual Power
0.6 388 0.8 0.800672
The sample size is for each group.
Often tests to distinguish between two binomial proportions are done in terms
of approximate normal tests, which are quite accurate for sample sizes this large
and for success probabilities not too near to $0$ or $1.$
Example of test of two proportions. Suppose that your results are $183$ in the first group and $241$ in the second.
Then Minitab's version of the one-sided test shows a highly significant difference with
a P-value near $0.$
Test and CI for Two Proportions
Sample X N Sample p
1 182 388 0.469072
2 241 388 0.621134
Difference = p (1) - p (2)
Estimate for difference: -0.152062
95% CI for difference: (-0.221312, -0.0828117)
Test for difference = 0 (vs ≠ 0):
Z = -4.30 P-Value = 0.000
Similar test in R: For comparison, the version of the test implemented in the R procedure 'prop.test'
gives the following result, also leading to rejection of the null hypothesis. (I use the version without continuity correction on account of the large sample size.)
prop.test(c(182,241), c(388,388), cor=F)
2-sample test for equality of proportions
without continuity correction
data: c(182, 241) out of c(388, 388)
X-squared = 18.091, df = 1, p-value = 2.106e-05
alternative hypothesis: two.sided
95 percent confidence interval:
prop 1 prop 2
Simulation of power. The following simulation in R with 'prop.test' shows that the power of the test
to distinguish between proportions $0.5$ and $0.6$ at the 5% level is roughly 80%.
pv = replicate(10^5, prop.test(rbinom(2,388,c(.5,.6)),c(388,388),cor=F)$p.val)
mean(pv <= .05)