2
$\begingroup$

Background on Two-Stage Designs
In clinical trials, we are often interested in the response rate $p$ for an experimental treatment. In a typical trial, we might expose $n$ patients to the treatment and observe the total number of responses $X$ in order to test the null hypothesis $H_0: p \leq p_0$ against a one-sided alternative $H_1: p > p_0$, where $p_0$ could be the response rate for a standard-of-care treatment. In order to limit patient exposure to an ineffective treatment, two-stage designs are popular: in the first stage, $n_1$ patients receive treatment and the number of responses $X_1$ is observed. If there are $r_1$ or fewer responses, the trial ends. If $X_1 > r_1$, a second stage is carried out: additional patients are enrolled until a total of $n$ patients have received treatment, and the total number of responses $X$ is observed. If there are more than $r$ total responses, the null hypothesis is rejected; otherwise, the null is not rejected and the new treatment is considered ineffective (or at least not superior to the standard of care). The parameters $n_1, r_1, n,$ and $r$ are chosen by the investigator before carrying out the study.

The Problem
While working on designing such a study, I have come across a phenomenon that I would like to better understand. Intuitively it seems that increasing the cut-off $r_1$ should decrease the type-1 error of the test. One might reason that for larger values of $r_1$, it is less likely that the study will proceed to the second stage, and thus less likely that one will ultimately reject the null. However, it appears that this is not always the case. For example, with the parameters $n_1 = 20, n = 30, r = 25,$ and $p_0 = 0.6$, I found the type-1 error to be the same (up to 9 digits) for $r = 10$ as for $r = 15$ (I got $\alpha = 0.001510074$). In fact, I get the same number for all values of $r_1 \leq 15$. See my code below.

My Question
So my question is, does it make sense that changing the value of $r_1$ does not affect the type-1 error in some cases? Can you help me refine my intuition to understand why this is true?

Code
Here is a quick function I have written to compute the type-1 error (as well as the power, for a given alternative, and the expected sample size) for two-stage designs.

##### Function to compute the type-1 error, power, and expected sample size for a
##### two-stage design. The parameters p0 and p1 are the response rates under the
##### null and alternative, respectively.

twoStage = function(n, n1, r1, r, p0, p1){

  # Type 1 error = P(reject H0 | p = p0) 
  #              = P(X1 > r1 && X > r | p = p0)]
  n2 = n - n1
  x1 = (r1 + 1):n1
  alpha = sum(dbinom(x1, n1, p0)*(1 - pbinom(r-x1, n2, p0)))

  # Power = P(reject H0 | p = p1) 
  #       = P(X1 > r1 && X > r | p = p1)]
  # Same as above, but with p = p1
  power = sum(dbinom(x1, n1, p1)*(1 - pbinom(r-x1, n2, p1)))

  # Expected number of subjects enrolled under H0
  expected.n = n1 + (1 - pbinom(r1, n1, p0))*n2

  return(cbind(n = n, n1 = n1, r1 = r1, r = r, p0 = p0, p1 = p1, alpha = alpha, 
  power =   power, expected.n = expected.n))

}

##### Example:

> twoStage(n=30, n1=20, r1=5, r=25, p0=0.6, p1=0.85)[,"alpha"]
  alpha 
0.001510074 
> twoStage(n=30, n1=20, r1=10, r=25, p0=0.6, p1=0.85)[,"alpha"]
  alpha 
0.001510074
$\endgroup$
  • $\begingroup$ You use the variables "p" and "p0" to denote the response rates. To avoid the possibility that some people will see this as a P value, I'd suggest using a different variable name. $\endgroup$ – Harvey Motulsky Jun 21 '14 at 16:13
1
$\begingroup$

Suppose $x_1 > r_1$ responses are observed in stage 1, and let $x_2$ be the number of responses observed in stage 2. Then $H_0$ will be rejected if $x_2 > r-x_1$. However, $x_2 \leq n - n_1$ (i.e. the number of patients in stage 2), which means that the inequality $x_1 > r + n_1 - n$ must be satisfied.

Letting $r^* = r + n_1 - n$, then the above implies $P(X_1 \leq r^*, X>r) = 0$, and hence $\alpha = P(X_1 > r_1, X>r) = P(X_1 > r^*, X>r)$ for $r_1 \leq r^*$.

Taking your example with parameters $n_1=20,n=30,r=25$ gives $r^* = 15$, and so the value of $\alpha$ will be the same for all values of $r_1 \leq 15$, as you observed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.