2
$\begingroup$

I think I have a simple question that I have not seen directly answered elsewhere, and as a stats beginner, I'm wary to translate answers that are not directly answering this question - apologies if it is viewed as duplicate.

My question is this - clinical trials are generally powered at 80% to show some level of treatment effect, but how can you calculate the minimum detectable treatment effect that would yield statistical significance?

For example let's say that a trial is 80% powered to show a treatment effect delta of 2.2 (e.g., the difference between treatment and control on a specific metric is 2.2)

Furthermore, we initially target an N of 210 for the entire trial and each arm is 1:1 randomized (105 per arm).

The standard deviation is assumed to be 8.05, statistical significance is defined as having a p-value < 0.05.

Applying the often used: enter image description here

We can solve for the treatment effect (the difference of the means) based on changing the other variables - specifically: enter image description here

Since we cannot change n, or Z(1-a) we can only reduce Z(1-b) and presumably assume a lower standard deviation.

Does this mean then that the true minimal difference that would be considered statistically significant, and thus a trial success, would be:enter image description here

In other words, simply letting Z(1-b) equal 0 for a powering of 50%?

Thus, on one hand, based on the example a minimum treatment effect could be 2.2 or 1.54 which could be a major difference in the likelihood of the trial being viewed as "successful" (showing stat sig) or not.

Many thanks!

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ Usually you don't "calculate" the minimum detactable effect size - rather you select such a size at the study design stage based on clinical considerations which are informed by prior research, etc. $\endgroup$
    – Isabella Ghement
    Jul 5, 2019 at 1:36
  • 1
    $\begingroup$ Yes but if I’m not designing the study but rather trying to infer what the sponsors of the trial selected would this be a reasonable approach? $\endgroup$
    – wingsoficarus116
    Jul 6, 2019 at 4:12
  • 1
    $\begingroup$ Why not ask the sponsors of the trial to disclose their choice of minimum detectable effect size? That information should not be a secret, unless something hanky-panky is going on. $\endgroup$
    – Isabella Ghement
    Jul 6, 2019 at 18:36
  • $\begingroup$ Because these disclosures are not always made and require contacting the sponsor? Can you please provide some insight into the mathematical approach here? $\endgroup$
    – wingsoficarus116
    Jul 7, 2019 at 18:52
  • 1
    $\begingroup$ I still maintain that you cannot "calculate" a minimally detactable effect size, but rather specify it based on subject matter considerations. As statisticians, we have to realize that there is a limit to what we can do - the clinical investigators we work with have a duty to step up to the plate and offer their expertise towards this specification. Specifying a minimally detectable effect size is not something that you should be expected to come up with, even under the best of circumstances. Maybe others here think differently, so I will let them wade in. $\endgroup$
    – Isabella Ghement
    Jul 7, 2019 at 20:19

1 Answer 1

Reset to default
1
$\begingroup$

The false positive error rate, power, sample size, and minimally detectable effect are all 4 tractably related in the first equation you specify above.

Your later derivation is correct. And you are correct that for a prespecified alpha level, desired power, fixed sample size, and known standard deviation, you can report the minimally detectable effect.

Mathematically, all these approaches should lead to equivalent findings. The goal is a judgement of whether or not the study is viable. It is efficient to focus on the "unknowns". It just so happens that it's rarely unknown what the effect should be. However, you should take care with prior knowledge about effect size. Preliminary studies are often hypothesis generating rather than hypothesis confirming, and tend to be selected on the basis of showing significant results. There will be regression to the mean.

The convention of 80% power is rarely appropriate for clinical studies. Considering the cost and time of running a trial, a 4/5 chance of correctly saying there's an effect if there is one is far too risky.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.