I am running a study to test the effectiveness of treatment A compared to treatment B. Given the study limitations, I am only able to recruit 200 participants into treatment A. The estimated incidence rate of disease in the treatment group is expected to be 20%. What is the sample size required for treatment B if I want to observed at least 5% (0.05) difference in incidence rate between treatment A and B (assuming type 1 error 0.05, power 80%)?


If I understand the question correctly, it will be very difficult to distinguish between an incidence rate of 0.2 in Group A with only $n_A = 200$ subjects, and an incidence rate of 0.25 in Group B, regardless of the number $n_B$ of subjects in Group B.

The estimate of $\hat p_A = X/n_a = X/200,$ where $X \sim \mathsf{Binom}(200, 0.2),$ has standard error $SD(\hat p_A) \approx \sqrt{.2(.8)/200} \approx 0.028.$ Thus, the margin of error of a 95% confidence interval for $p_A$ will be about $\pm 1.96(0.028) \approx 0.055.$ However, you want to detect a difference of size $0.05$ between $p_A$ and $p_B,$ which seems difficult even if $p_B$ were known exactly.

In R, the procedure prop.test is a test of two proportions. If $n_A = n_B = 500,\,$ $X \sim \mathsf{Binom}(500, .2),$ and $Y \sim \mathsf{Binom}(500, .25),$ then we might get $\hat p_A = 0.222, \hat p_b = 0.236.$

x = rbinom(1, 500, .2);  x
[1] 111
y = rbinom(1, 500, .25); y
[1] 118
x/500;  y/500
[1] 0.222
[1] 0.236

Then prop.test gives the following output, showing no significant difference (P-value $0.60 > 0.05 = 5\%).$ [Continuity correction is suppressed on account of sample sizes exceeding 100.]

prop.test(c(111,118), c(500,500), cor=F)

        2-sample test for equality of proportions 
        without continuity correction

data:  c(111, 118) out of c(500, 500)
X-squared = 0.27753, df = 1, p-value = 0.5983
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.06607901  0.03807901
sample estimates:
prop 1 prop 2 
 0.222  0.236 

A simulation of many such cases with $n_A = n_b = 500$ shows that the null hypothesis is rejected in less than half of iterations. (That is, power less than 50%.)

By contrast, if $n_A = n_B = 1200,$ then one has power just above $80\%$ detecting a difference between $p_A = 0.20$ and $p_B = 0.25.$

na = nb = 1200
                         rbinom(1,nb,.25)), c(na,nb))$p.val)
mean(pv <= .05)
[1] 0.823413
| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.