3
$\begingroup$

I've to calculate the sample size for a superiority trial.

I've found many online calculator, but each one gives a different result and I don't understand why.

For istance, this one: https://riskcalc.org/samplesize/ -> Superiority Trial -> dichotomous outcome -> proportion: I don't understand the difference between the superiority margin and the difference between pt and PC.

In other calculators, like this one, the superiority margin is not required : https://app.sampsize.org.uk/ superiority -> parallel -> binary -> sample size

why? what is the correct procedure?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

An appropriate method depends on the details of your planned analysis. You have to look carefully at the documentation of these calculators to see if they are fit for your purpose.

The first link (https://riskcalc.org/samplesize/) says it relies on Chow et al. (2017). In short, the method the calculator is using may be appropriate when you plan to use a two-sample Z-test for proportions with unpooled variance, possibly with an option for defining superiority by a margin. If you define a margin $\delta > 0$, in the case of a one-sided test, it means that you'll be using a test where your null hypothesis is no longer $p_t - p_c \leq 0$ but $p_t - p_c \leq \delta$.

Say the control group has a value $p_c = 0.6$. You want to be able to detect an effect such as the treatment group has a value $p_t = 0.75$, with a power of 0.8 and an alpha of 0.025. If the null hypothesis ($p_t-p_c \leq 0$) is true, then the Z-test will reject the null at most 2.5% of times if you were to repeat the study a lot of times. If $p_t = 0.75$, the test will reject the null 80% of times (and more than that if $p_t > 0.75$). If the real value of $p_t$ is anywhere between 0.6 and 0.75, the power will be smaller than 0.8, approaching 0.025 as $p_t$ approaches 0.6. So far so good. In this case, you'd enter the corresponding values in the calculator: $p_t=0.75$, $p_c=0.6$, etc., and importantly, a margin $\delta = 0$.

However, for some reasons specific to your study, you consider that any value of the treatment such as $p_t \leq 0.7$ is ignorable and should be treated the same way as if $p_t \leq 0.6$. Therefore, you want your test to have a maximum rejection rate of 2.5% for any value $\leq 0.7$. You plan to use a test configured for this purpose, so it will be able to treat this range of values as the null hypothesis. In this example, your margin of superiority would be $\delta = 0.7-0.6 =0.1$, which is the value you should enter in the "margin" field of the sample size calculator. But since you're conducting power analysis, you also have to determine an effect size you want to be able to detect with a certain power. In this example, the effect size defined earlier is $p_t=0.75$ (not $p_t=0.7$), so you have to enter 0.75 in the $p_t$ field of the calculator (in addition to entering 0.1 in the margin field).

Now, regarding the second link (https://app.sampsize.org.uk/), it indirectly mentions Julious & Campbell (2017). Reading their paper, the method they suggest is adequate if you plan to use a two-sample Z-test for proportions with pooled variance. The calculator doesn't have an option for defining a margin, i.e. there's no possibility to define a range of values for the null hypothesis like the previous calculator. So essentially it's like you were defining a margin of 0 in the first calculator. However, there's still a difference between the two calculators even when we use a margin of 0, because the first link assumes unpooled variance for the Z-test, while the second link assumes pooled variance. This difference may impact the required sample size minorly or majorly, depending on the situation. You can probably find some discussion about the difference between pooled and unpooled variance on this very website (e.g. Is there a reference that legitimises the use of the unpooled z-test to compare two proportions?).

So essentially the method to use depends on the analysis you plan to conduct and the tests you'll be using. If you give more details about your analysis, maybe someone will be able to help you further. It might be sensible to conduct power analysis by simulation, as it could give you more flexibility than pre-packaged calculators, that do not necessarily take into account all the details of your analysis.

References

Chow S-C, Shao J, Wang H, Lokhnygina Y. Sample Size Calculations in Clinical Research. Third ed: Chapman and Hall/CRC; 2017.

Julious, S.A. and Campbell, M.J. (2012) Tutorial in biostatistics: sample sizes for parallel group clinical trials with binary data. Statistics in Medicine , 31 (24). pp. 2904-2936. ISSN 0277-6715 https://eprints.whiterose.ac.uk/145472/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.