5
$\begingroup$

Most clinical trials I see have a fixed sample size. In some cases they have prior data that allows estimating the effect size and the variance or distribution of values, and calculate the sample size from that for a certain power. In other cases it is just a guess.

Why wouldn't people run a clinical trial in which the sample size was determined during the trial? (for example, by increasing it until the confidence interval narrowed to a certain size specified in advance) Is there any reason this would not be a valid design? Are there any examples of trials like that, and any references for designing a trial like that?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

Ideally that's the point of a Phase II trial. Results from these studies, often single-arm in design, are used for power calculations. Sometimes they experiment with dosing and eligibility criteria, the more moving parts in a Phase II study, the more of a gamble a Phase III study will be.

If a compound is showing to be promising a Data Monitoring Committee might recommend increasing enrollment or decreasing it appropriately. Sometimes it's about the risk of harm. If a compound is underpowered because the effect is not as powered as it was hoped to be, the DMC may end the study since the study subjects, by virtue of participating in the study, are exposing themselves to risk. Studies cannot go on perpetually as a matter of ethics.

Indeed there is a whole field of sequential adaptive trials that allows researchers to seamlessly transition from Phase II to Phase III studies. The statistical software package SeqTrial in S+ from Scott Emerson allows you to perform sample size calculations for a variety of alpha spending rules and effect sizes.

The FDA's overreliance on "traditional" statistics is pretty against it, as it can affect the integrity of findings. It's actually a good principle in this case, and Tom Fleming has rallied against it in his paper "Discerning Hype From Substance." Basically, collating Phase II and Phase III study findings is rarely if ever appropriate, even when the protocols are similar (identical) between II and III. This is because the Phase III study only happened because Phase II looks/looked promising. So selection bias will affect the validity of those aggregated findings.

Improve this answer
$\endgroup$
5
  • $\begingroup$ "it can affect the integrity of findings" - I get the multiple comparisons problem, but aren't there well-established ways to control for that? I'm not really talking about continuing a study until it gets a significant result, but rather until the uncertainty about result size drops enough; this wouldn't seem to be as affected by multiple comparisons. $\endgroup$
    – Alex I
    Commented Jul 1, 2015 at 4:58
  • $\begingroup$ could you also tell me what the reputation/credibility issues are (if any) with sequential trials? Is a well-designed and ran sequential trial considered strong evidence by most people? Do people not usually do this because it is too complicated or because it is not credible enough? $\endgroup$
    – Alex I
    Commented Jul 1, 2015 at 5:04
  • $\begingroup$ I think if you believe in maximum likelihood principle or Bayesian, adjustable sample size for a clinical trial is OK. $\endgroup$
    – Deep North
    Commented Jul 1, 2015 at 11:42
  • 1
    $\begingroup$ @AlexI The integrity of the study is not simply a power issue. When you conduct a trial, you are exposing people to harm: new therapies are either superior or inferior, so you need to get your answer as quickly as possible. Sample size calculations suggest that, in the case of "borderline" results, you need to increase sample size when sometimes you need to call the trial off because the existing therapy works just fine. Noninferiority is another issue altogether. $\endgroup$
    – AdamO
    Commented Jul 1, 2015 at 16:40
  • $\begingroup$ @AlexI as I alluded to, Phase III trials happen as a consequence of a promising Phase II trial. Sequential trials are largely interested in combining their respective processes. Suppose, however, I have 20 random compounds being administered to 20 groups of 20 rats with pancreatic cancer. One particular group has complete remission from cancer. Very promising indeed, so we do a Phase III trial. If I include the original 20 survivors in my analysis sample for a phase 3 trial, it will be biased no matter what I do. $\endgroup$
    – AdamO
    Commented Jul 1, 2015 at 16:43
3
$\begingroup$

I think AdamO's answer is great, but I think it's also worth emphasizing out that this adaptive sample size design is how many (maybe even most? I've done theoretical work during internships at pharm companies, but can't say I've ever planned a real study...) clinical trials are run.

That is to say, if a sequential design is used, initial patients are recruited and treated. Part way through the study, the currently collected data gets analyzed. Three possible actions can occur at this point: the data may show a statistically significant result and the study will be stopped because efficacy has been demonstrated, the data many statistically significantly show that there is no strong effect (for example, the upper end of the confidence interval is below some clinically significant threshold) and the study is stopped due to futility or the data is not yet conclusive (i.e. both a clinically significant effect and a clinically insignificant effect are contained in the confidence interval) in which more data will be collected. So you can see that in this case, the sample size is not fixed.

An important note about this: you can't just run a standard test each time you "check" your data, otherwise you are doing multiple comparisons! Because the test statistics at different times should be positively correlated, it's not as big an issue as standard multiple comparison issues, but it still should be addressed for proper inference. Clinical trials, being regulated by the FDA, must state a plan for how they will address this (as @AdamO points out, SeqTrial provides software for this). However, often times academic researchers, not being regulated by the FDA, will continue to collect data until they find significance without adjusting for the fact that they are doing several comparisons. It's not the biggest abuse of statistical practice in research, but it still is an abuse.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.