Sample size calculation with fixed sample size in one group

Question

I am running a study to test the effectiveness of treatment A compared to treatment B. Given the study limitations, I am only able to recruit 200 participants into treatment A. The estimated incidence rate of disease in the treatment group is expected to be 20%. What is the sample size required for treatment B if I want to observed at least 5% (0.05) difference in incidence rate between treatment A and B (assuming type 1 error 0.05, power 80%)?

Answer 1

If I understand the question correctly, it will be very difficult to distinguish between an incidence rate of 0.2 in Group A with only $n_A = 200$ subjects, and an incidence rate of 0.25 in Group B, regardless of the number $n_B$ of subjects in Group B.

The estimate of $\hat p_A = X/n_a = X/200,$ where $X \sim \mathsf{Binom}(200, 0.2),$ has standard error $SD(\hat p_A) \approx \sqrt{.2(.8)/200} \approx 0.028.$ Thus, the margin of error of a 95% confidence interval for $p_A$ will be about $\pm 1.96(0.028) \approx 0.055.$ However, you want to detect a difference of size $0.05$ between $p_A$ and $p_B,$ which seems difficult even if $p_B$ were known exactly.

In R, the procedure prop.test is a test of two proportions. If $n_A = n_B = 500,\,$ $X \sim \mathsf{Binom}(500, .2),$ and $Y \sim \mathsf{Binom}(500, .25),$ then we might get $\hat p_A = 0.222, \hat p_b = 0.236.$

set.seed(921)
x = rbinom(1, 500, .2);  x
[1] 111
y = rbinom(1, 500, .25); y
[1] 118
x/500;  y/500
[1] 0.222
[1] 0.236

Then prop.test gives the following output, showing no significant difference (P-value $0.60 > 0.05 = 5\%).$ [Continuity correction is suppressed on account of sample sizes exceeding 100.]

prop.test(c(111,118), c(500,500), cor=F)

        2-sample test for equality of proportions 
        without continuity correction

data:  c(111, 118) out of c(500, 500)
X-squared = 0.27753, df = 1, p-value = 0.5983
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.06607901  0.03807901
sample estimates:
prop 1 prop 2 
 0.222  0.236 

A simulation of many such cases with $n_A = n_b = 500$ shows that the null hypothesis is rejected in less than half of iterations. (That is, power less than 50%.)

By contrast, if $n_A = n_B = 1200,$ then one has power just above $80\%$ detecting a difference between $p_A = 0.20$ and $p_B = 0.25.$

set.seed(2020)
na = nb = 1200
pv=replicate(10^6,
             prop.test(c(rbinom(1,na,.2), 
                         rbinom(1,nb,.25)), c(na,nb))$p.val)
mean(pv <= .05)
[1] 0.823413