We got two types of treatment (type 1 and type 2). In our opinion, type 2 treatment has a higher performance and we would like to show this with research.
We assume that type 1 treatment has a maximum of 60% success, and type 2 treatment is 80% successful. We are interested in getting the smallest sample size that the test power will be 0.8. We must decide which test is appropriate and get the smallest sample size with simulations.

What I have so far:

Since this is a proportions test, I asume that the null hypothesis will be: $H_0: p_1 = p_2$ and the alternative hypothesis will be: $H_A: p_1 = 0.6, p_2 = 0.8$. Distribution of every person in type 1 treatment is Ber($p_1$) and in type 2 treatment Ber($p_2$). Therefore, the distribution of treatment success is Binomial.

For comparing two proportions I used the Z test ($Z = \frac{p_1 - p_2 - 0}{\sqrt{p(1-p)(\frac{1}{n1} + \frac{1}{n2})}}$ and got two critical values: 1.96 and -1.96.

I generated 1000 samples for each group under the alternative hypothesis (type 1 and type 2) and calculated test statistics for each sample. Finally, I checked how often were the test statistics in the rejection region. The first $n$ with $80\%$ of rejection is the sample size I am looking for.

My question:

Is my approach correct? Is there a better, more accurate way of solving this problem? Should I use two sample sizes i.e. should I loop over $n1$ and $n2$ for treatment one and two respectively?

  • $\begingroup$ you could take a look at power.prop.test() in R ... $\endgroup$
    – Ben Bolker
    Apr 1, 2019 at 2:20

1 Answer 1


I think there is a straightforward approach to solve what you are asking (if I understood correctly what you want). There are formulas to computed the sample size from the data you presented. I will use one formula from "Sample Size Calculations in Clinical Research Second Edition" by Shein-Chung Chow (Ed.).

The sample size can be computed from: $$ n = \frac{r+1}{r} \left(\frac{(z_a + z_b ) s}{D} \right)^2 $$ With $r$ being the ratio between the largest and the smallest group size. If the groups are equally sized, then $r=1$. In your case, the difference between proportions is $D=0.8-0.6=0.2$. The pooled standard deviation following the null hypothesis is $s = \sqrt{p(1-p)} = 0.49$, for your example.
The values of $z_a$ and $z_b$ correspond to the cut-offs for the $5\%$ significance and the $80\%$ power, thus $z_a=1.96$ and $z_b=0.84$.

Plugging all of these into the formula, gives: $$ n = 94.12 \ . $$ So, your sample size in the end of the treatment should be around 94 individuals. Remember to start with more people because there may be some drop-offs.

  • $\begingroup$ Thank you for your answer. I have some questions about the formula from the book. Is there any chance you could tell me on which page/chapter in the book is this formula derived? As I have trouble finding it. $\endgroup$
    – Ravonrip
    Apr 2, 2019 at 11:38
  • $\begingroup$ Chapter 4 has several formulas to compare proportions on different studies. I think you might want the formula from 4.2.1 (I used a more pessimistic estimate of $s$, based on the probability closer to 50% and gave you the total sample size, where in the book the sample size of each group is given). $\endgroup$ Apr 2, 2019 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.