I was asked to perform a sample size calculation for a study with a single arm and a binary outcome. They have given me an assumed proportion, $p$, and need the upper 95% confidence interval to be above a given threshold, $p_0$.

I presented numbers using the Wilson and the Jeffreys intervals (and their modified versions), and showed that as the sample size increases that power will jump up and slowly decrease then jump up again and recommended that we pick the lowest sample size where all larger sample sizes would have at least the desired power (80%).

The numbers I presented were apparently higher than what was wanted, and someone pointed to this paper, and suggested that I run the power calculations as described there. The paper presented results using the exact (Clopper-Pearson) interval. I verified that the exact interval would give the results described (202 out of 202 yields an upper 95% confidence bound of 0.9853, which they reported to be 98.5%, and 198 out of 199 gives 0.9764, which they reported as 97.6%). So far, so good. Then I get to the part where they described their sample size calculation:

"We anticipated a cure rate of 97% and lower 1-sided 95% CI bound of ≥95% and allowed for a 10% dropout rate. The target sample size was 250 infected participants per group."

Keep in mind that when it mentions "per group", the study has two regimens and they are running two one-sample proportion tests. We are not comparing groups.

The problem is that when I run the power for 225 subjects (after accounting for 10% dropout) using Clopper-Pearson and the parameters they describe, power calculations I have run in both SAS and R estimate only 33% power. Here's the SAS code:

proc power;
  onesamplefreq test=exact
  nullproportion = 0.95
  proportion = 0.97
  ntotal = 225
  power = .;

Nowhere in the report does it say what level of power was used, but 33% seems particularly bold. The study team is expecting me to come back with a sample size estimate that is below what was used in this study. I believe that is wrong, but I would like to be able to figure out how they came up with these numbers so I can explain where they went wrong and why we should go with my numbers. I'll recommend my numbers regardless but it would be better if I could explain where the team in this study went wrong.

Can anyone help me figure out how they arrived at 225 subjects?

  • $\begingroup$ I might guess they made a mistake, and reviewers didn't see it. $\endgroup$ – Jeremy Miles May 20 '15 at 19:50
  • $\begingroup$ Hard to say, there is not much else to reverse engineer. Another guess would be that there was a default in the program the original authors used and that the null proportion is 0.5. A reduction in only 2% is really small, if the null proportion was say 0.93 instead of 0.95 the power is above 0.8 (Stata code power oneproportion .93 .97, test(binomial) n(225) critvalues onesided). $\endgroup$ – Andy W May 21 '15 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.