How does one perform a conditional power analysis?

Question

Suppose a test statistic $Z$ will be compared to a standard normal distribution to evaluate whether a $p$-value achieves a statistically significant result at the $\alpha$-level. Based on an anticipated effect size $\phi$, a sample size of $n$ is anticipated to have power $1-\beta$.

At an interim analysis, $m < n$ subjects are recruited into the trial and the interim effect size is some value $\hat{\phi}_m < \phi$. Now we believe the power is not as high as anticipated. We wish to perform a conditional power analysis; What is the probability of achieving a statistically significant result after recruiting all $n$ subjects? What if we use as a new effect size $\phi$ or $\hat{\phi}$?

Answer 1

Before we proceed, let's make some assumptions clear:

• Assumptions for Conditional Power Analysis:

The sample size for the entire study is $n$. The interim analysis is performed after $m$ subjects, with $m < n$. $\hat{\phi}_m$ is the observed effect size at the interim analysis. $\phi$ is the originally anticipated effect size. $Z$ is the test statistic that follows a standard normal distribution.

• Conditional Power Computation:

To compute the conditional power, we first need to understand what future effect size we're assuming:

Scenario 1 - Using Original Anticipated Effect Size $\phi$: Assume that in the future, the effect size will revert to the originally anticipated $\phi$. In this scenario, you would adjust the test statistic $Z$ based on the interim results and compute the power for the remaining $n - m$ subjects assuming the effect size will be $\phi$.

Scenario 2 - Using Observed Effect Size $\hat{\phi}_m$: Assume that the effect size observed in the interim analysis will persist for the duration of the study. Here, the computation would consider the power for the remaining $n - m$ subjects using the effect size $\hat{\phi}_m$.

• Calculation:

Conditional power ($CP$) can be represented as:

$CP = P\left(Z > z_\alpha + \sqrt{n}(\phi - \hat{\phi}_m)) | \text{data up to interim analysis} \right)$

Where:

$z_\alpha$ is the critical value from the standard normal distribution for the given significance level $\alpha$. The key point to grasp here is that the conditional power depends heavily on the assumptions made about the future effect size. If you assume that future effect sizes will resemble the interim observed effect size, and this is less than the originally anticipated effect size, then the conditional power will generally be less than the original power $1-\beta$. Conversely, if you believe that the study will revert to the originally anticipated effect size, the conditional power might be closer to $1-\beta$.

It's important to approach conditional power with caution, as it is contingent on the assumptions made. If these assumptions do not hold true, then the actual power may differ significantly from the computed conditional power.

For further details I would recommend the following paper:

Edwards, J.M., Walters, S.J. & Julious, S.A. A retrospective analysis of conditional power assumptions in clinical trials with continuous or binary endpoints. Trials 24, 215 (2023). https://doi.org/10.1186/s13063-023-07202-6