Sample size for 2-arm 2-stage trial: R package Trialsize vs rpact vs gsDesign

Question

Which sample size calculation is correct?

Intended design:

• Superiority two-arm trial between drug A (intervention) vs drug B (comparator),
• outcome is binomial (response or no response to treatment),
• randomised,
• alpha 0.05,
• beta 0.2
• one-sided (superiority) testing
• drug A (intervention) expected response 40%
• drug B (comparator) expected response 15%
• superiority margin 11% i.e H0: intervention response rate - comparator response rate = 0.11
• allocation ratio 1
• interim analysis once, at 30% information rate.

#install.packages("TrialSize")
library(TrialSize)

# p1: test
# p2: reference
# k: allocation ratio
# delta: expected difference in proportion
# margin: non-inferiority or superiority margin
# H0: delta =< margin, HA: delta > margin
# but this is a fixed design with no interim analysis
ss1 <- TwoSampleProportion.NIS(alpha = 0.05, beta = 0.2, p1 = 0.4, p2 = 0.15, k = 1, delta = 0.25, margin = 0.11)
paste("Total sample size: ", round(ss1))
ss1.10 <- ss1/0.9
paste("Total sample size including 10% dropout: ", round(ss1.10))

Yields sample size of 116

library(rpact)
# pi1 intervention treatment
# pi2 comparator treatment
## interim ana at 30% information rate
## superiority margin: H0: pi(1) - pi(2) = 0.11
design <- getDesignGroupSequential(typeOfDesign = "OF", informationRates = c(0.30, 1), alpha = 0.05, beta = 0.2, sided = 1, tolerance = 1e-08)
designPlan <- getSampleSizeRates(design, riskRatio = FALSE, 
                                 thetaH0 = 0.11, #superiority margin
  normalApproximation = TRUE, pi1 = 0.40, pi2 = 0.15, groups = 2, allocationRatioPlanned =
  1)
summary(designPlan)
d2 <- as.data.frame(designPlan)
ss2 <- d2[1,12]
paste("Total sample size: ", round(ss2))
ss2.10 <- ss2/0.9
paste("Total sample size including 10% dropout: ", round(ss2.10))

Yields sample size of 239

library(gsDesign)
# Design assumptions

# Control rate for endpoint (comparator drug)
p1 <- 0.15
# Experimental rate (intervention drug)
p2 <- 0.4
# Difference in rates under H0
delta0 <- -0.11
# 1-sided Type I error
alpha <- 0.05
# Type 2 error (1 - targeted power)
beta <- 0.2
# Randomization ratio
ratio <- 1

# Number of analyses
k <- 2
# See gsDesign() help for description of boundary types
test.type <- 1
# If test.type = 5 or 6, this sets maximum spending for futility
# under the null hypothesis. Otherwise, this is ignored.
astar <- 0
# Timing (information fraction) at interim analyses
timing <- c(0.3,1) ############# information rate
# Efficacy bound spending function
sfu <- sfLDOF
# Upper bound spending function parameters, if any
sfupar <- c(0)
# Lower bound spending function, if used (test.type > 2)
sfl <- sfLDOF
# Lower bound spending function parameters, if any
sflpar <- c(0)
# Difference in event rates under null hypothesis
delta0 <- -0.11
# Difference in event rates under alternate hypothesis (< delta)
delta1 <- -0.25

# Derive group sequential design
x <- gsDesign(
  k = k,
  test.type = test.type,
  alpha = alpha, # 1-sided Type I error
  beta = beta,   # Type II error (1 - Power)
  astar = astar,
  timing = timing,
  sfu = sfu,
  sfupar = sfupar,
  sfl = sfl,
  sflpar = sflpar,
  delta1 = delta1,
  delta0 = delta0,
  endpoint = "Binomial",
  n.fix = n # Fixed design sample size from nBinomial above
)

gsBoundSummary(
  x,
  deltaname = "risk difference",
  digits = 4,
  ddigits = 2,
  tdigits = 1
)

cat(summary(x))

Yields sample size of 77

Answer 1

Sample size calculations for clinical trials, particularly superiority trials, can be complicated and is dependent on a number of characteristics and assumptions. In your scenario, you calculated the sample size for a two-arm trial comparing two medications with a binomial outcome using three distinct methods. Each strategy takes into account various aspects of trial design and statistical concerns. Here's a quick rundown of why these strategies could produce different results:

TrialSize package for R : This approach computes the sample size for a non-inferiority/superiority study with a fixed design that does not take into account interim analysis. It takes into account alpha, beta, projected response rates, and superiority margin. This method is simple, but it does not account for the intricacies of group sequential designs or interim analyses, which can have a considerable impact on sample size.

rpact package for R : This method employs a group sequential design with a 30% information rate interim analysis (a pre-planned point for interim analysis when a certain portion - in this case, 30% - of the total data required for the trial's final analysis has been gathered.). Interim analyses in group sequential designs enable for the trial to be stopped early for efficacy, futility, or harm, influencing the overall sample size. This approach is more sophisticated and practical for many clinical trials since it accounts for the possibility of early termination of the experiment, resulting in a higher sample size.

gsDesign package for R: This method also employs a group sequential design, but the assumptions or spending functions for the interim analysis differ. The expenditure function controls how the alpha level is distributed between intermediate analyses, which influences the total sample size. Different assumptions about the expenditure function or intermediate analysis settings can result in various sample size computations.

Which One is Correct?

• Depends on Trial Design and Assumptions: Each strategy is deemed "correct" based on its underlying assumptions and the particular trial design it is employed for. The optimal approach is contingent upon the specific configuration of your experiment and the objectives you seek to accomplish.

• Regulatory Requirements and Trial Objectives :**The sample size calculation for a real-world clinical trial must be aligned with the trial's objectives, the statistical power required, ethical issues, and regulatory guidelines.

Consult with a Statistician: Given the discrepancies in the computations, consulting with a statistician or clinical trial design expert is strongly advised. They may assist you in determining the best approach for your individual trial by taking into account all important elements, such as the possibility of interim analysis and the ramifications of different sequential designs.

Alternative Bayesian approach.

As mentioned by Frank Harrell in the first comment on the question, note that the above three R packages use frequentist approaches to the problem. In the frequentist framework, there is a higher likelihood of observing extreme data when examining data more frequently. In the Bayesian framework, the likelihood of the truthfulness of an assertion is unaffected by the number of times it has been previously made. Bayesian analysis in sequential testing does not involve multiplicity corrections. These methods tend to employ the same formulas for all interim analyses as they do for the final analysis.

Adopting a Bayesian approach in conjunction with group sequential designs can offer additional flexibility and insights. The R package gsbdesign is specifically tailored for implementing Bayesian group sequential designs. In this framework, Bayesian statistics are used to update the probability distributions of the treatment's effectiveness as data is collected throughout the trial. This approach allows for a more dynamic and continuous assessment of the data, incorporating prior knowledge and real-time data analysis, potentially leading to smaller sample sizes needed. Bayesian methods can be particularly advantageous in interim analyses, such as at the 30% information rate, where they provide a probabilistic interpretation of the results. This interpretation can be crucial for decision-making, especially when considering early stopping for efficacy, futility or harm. By integrating Bayesian methods, researchers can gain a more nuanced understanding of the treatment's effects at each stage of the trial, potentially leading to more informed and timely decisions about the trial's continuation or modification. Here is a fantastic introduction to Bayesian sequential analysis by Frank Harrell.