Usually, we calculate the sample size needed for a clinical trial based on a former clinical trial which shows there is a significant effect. What if there is no such a trial? What if there is no information? How do we decide the sample size? Any idea?
1 Answer
It would be unusual (if not irresponsible) for a clinical trial to be conducted on a new drug in a complete vacuum of information. You need to try to find a suitable sample size for such a trial, given the information you do have along with customary practice for clinical trials.
Suppose the proposed clinical trial is for a new drug that has determined to be safe to use on human subjects. Now it is time to assess its effectiveness.
An existing drug for the same condition gives favorable results with 70% of subjects, and you consider this "cure rate" to be accurate. The new drug will be considered worth additional attention, if it raises the cure rate to 80%.
Suppose you want to work at the 5% level of significance and you want power (probability of detection of increased cure rate as above) to be about 95%. Then you have enough information, even if partially speculative, to determine a suitable sample size.
Here is a 'power and sample size' procedure from a recent release of Minitab. Other statistical software programs and some online 'calculators' work similarly. You will need about 800 suitable subjects randomized into two groups of 400 each, one receiving the old drug and one receiving the new.
Power and Sample Size
Test for Two Proportions
Testing comparison p = baseline p (versus >)
Calculating power for baseline p = 0.7
α = 0.05
Sample Target
Comparison p Size Power Actual Power
0.8 404 0.95 0.950370
The sample size is for each group.
There are several versions of tests comparing two
proportions. Here is an example of results from prop.test
in R for a trial with 320 cures for the new drug and 280 for
the old drug out of 400 in each group. (Of course you can't
expect sample proportions will always come so close to
theoretical population proportions).
prop.test(c(320, 280), c(400,400), alt="g", cor=F)
2-sample test for equality of proportions
without continuity correction
data: c(320, 280) out of c(400, 400)
X-squared = 10.667, df = 1, p-value = 0.0005454
alternative hypothesis: greater
95 percent confidence interval:
0.04997373 1.00000000
sample estimates:
prop 1 prop 2
0.8 0.7
A simulation in R of $100\,000$ trials with binomial data can be used to determine whether a trial with 400 subjects in each group will perform as expected. The result is a simulated power very nearly 95%.
set.seed(2022)
pv = replicate( 10^5,
prop.test(c(rbinom(1,400,.8),rbinom(1,400,.7)),
c(400,400), alt="g", cor=F)$p.val )
mean(pv <= 0.05)
[1] 0.94826 # aprx power
-
1$\begingroup$ Thank you for the answer. So you basically use simulation to decide the sample size now. Am I understanding it right? $\endgroup$ May 23, 2022 at 3:48
-
$\begingroup$ The Minitab and R 'power and sample size' procedures use somewhat complicated formulas. Also the power curve is from a formula. // To verify that the slightly different versions of the test in Minitab and R require the same sample size for the same task, I did the simulation in R at the end, // Only_the last few lines at the very end of my answer use simulation. Even though it requires a lot of computation, the simulation at the end is very easy to program and may be more transparent to a beginner who is familiar with simulation than the analytic formulas. $\endgroup$– BruceETMay 23, 2022 at 5:54
-
$\begingroup$ Thank you. Although could you elaborate on the "cure rate" you assumed? In R, it said p1 and p2 are "probability in one group", and I wonder what this probability is. $\endgroup$ May 23, 2022 at 13:28
-
$\begingroup$ Assuming the drug is supposed to cure a disease or condition, then 'cure rate' is probability of success. You hope it will be higher for the new drug. // No idea what your second sentence refers to. Throughout, $p_1$ and $p_2$ are 'success' probabilities for Groups 1 and 2, respectively. $\endgroup$– BruceETMay 23, 2022 at 16:45