1
$\begingroup$

I'd be grateful for thoughts on this problem.

I'm considering a sample size calculation for a two-armed (controlled) trial with an 8-point ordinal outcome. Patients present with brain injury, are randomised to recieve either the treatment or control, and are followed up at 6 months using the standard outcome scale for this condition which is an 8-point ordinal scoring system, the Glasgow Outcome Score-Extended.

In the literature I've seen a variety of approaches and I'm debating which would be the better option, or if anyone has any better ideas. (Most of what I've read about sample size calculation seems to rely on having a mean as a measure of central tendency which clearly wouldn't be appropriate here). I'm a medic, new to research and still getting my head around some of the concepts!

  1. Using the outcome score as it is (not sure how this would work or how I'd be able to work out the power to detect if one population tended to have a higher score than the other)
  2. Dividing the score into dichotomous groups (1-4 being a poor outcome/ 5-8 being a favourable outcome), and comparing the odds of having a poor outcome (this is the most common approach I've seen in the literature so far)
  3. Using a sliding scale for a dichotomous outcome by using a validated prognostic scoring system at presentation (the IMPACT score), and then analysing if each patient had a dichotomised poor or favorable outcome based on their predicted prognosis (this has been used in a couple of large trials in this area)
  4. Going even further than that and creating a scaled outcome for each individual patient (for example, a patient predicted to have a 0.56-1/1 probability of a poor outcome would score 0 for an outcome score of 4, +1 for a score of 5, +2 for a score of 6 etc and -1 for a 3, -2 for a 2 etc. (This has been described in the literature but I haven't seen any trials use this methodology yet, but there was validation when this methodology was applied to the dataset of a large trial)

My working so far has been towards 2 as that's the simpler test. 2 to 4 seem to go up rapidly in complexity but I'm a bit wary about how valid a scoring system such as 4 would be. However 3 makes a lot more clinical sense to me - not all patients are equally severe and an outcome score of e.g. 5 would mean very different things six months after a more mild injury when compared to someone with a very severe injury.

$\endgroup$
4
  • $\begingroup$ Welcome to Cross Validated! Will you be only evaluating this ordinal outcome at 6 months, or is there a baseline evaluation prior to treatment/control? Do you have any estimate of the distribution of outcome values under control treatment from your experience or the literature? Also, are you aware of ordinal logistic regression? $\endgroup$
    – EdM
    Aug 23, 2022 at 20:05
  • $\begingroup$ Thanks @EdM. The standard for this outcome is an assessment at 6 months only (setting is traumatic brain injury, score is the Glasgow Outcome Score-Extended, and in the rest of the literature this is usually assessed no earlier than 6 months - which makes clinical sense as the brain injury evolves over time and with rehabilitation which is why when comparing against other studies, this is taken as a standard time point). There will be a baseline evaluation, but this will be with a different score (IMPACT) which gives a probability of a poor outcome... $\endgroup$
    – docBen
    Aug 24, 2022 at 6:36
  • $\begingroup$ ... For the distribution of outcome values - yes, as this is a standard measure at a standard time, there are thankfully several large trials and registries which I can use to get a distribution of similar patients. I haven't heard of ordinal logistic regression - but having a look now and trying to get my head around it!!! $\endgroup$
    – docBen
    Aug 24, 2022 at 6:40
  • $\begingroup$ From my reading on ordinal logistic regression, I believe that I would have to make a proportional odds assumption? If I understand this, this should mean that the populations differ to the same degree in each ordinal category, and if they don't then I can't use this test. Would this be a reasonable assumption for a power calculation? (Obviously it may not be for my final data and I would have to test it at the time) $\endgroup$
    – docBen
    Aug 24, 2022 at 7:04

1 Answer 1

0
$\begingroup$

If you don't have to correct for covariates and you just want to compare two groups at 6 months after randomization, then with an 8-point ordinal outcome you could use the Wilcoxon-Mann-Whitney test (WMW) to evaluate the stochastic ordering between treatment groups, that is, whether a randomly chosen member of one group is likely to have a greater value than a randomly chosen member of the other. If you do need to adjust for covariates like the IMPACT score at presentation, then an ordinal logistic regression model is the direct extension of the WMW test.

This web post from Frank Harrell summarizes the issues nicely, and demonstrates that you can get useful results even if the proportional-odds (PO) assumption is relaxed. In particular, with respect to your suggestion to dichotomize outcomes:

a unified PO model analysis is decidedly better than turning to inefficient and arbitrary analyses of dichotomized values of Y.

This page outlines approaches to power/sample-size calculations for the WMW test. It's probably most reliable to make the estimate based on multiple sampling from simulated data.

You start with the known distribution of Glasgow scores under control conditions, and you make specific assumptions about how you expect the new treatment will change the distribution of scores. Simulate large numbers of both groups under those assumptions, and take repeated samples of different sizes to determine the sample size needed to detect your assumed difference at the desired power and significance criteria.

Simulation has a particular advantage if you will be controlling for covariates, as it allows you to consider the distributions of covariate values and their associations with outcomes and treatment effects. Then you can evaluate multiple scenarios about the effect of treatment to better ensure that your study will be large enough to provide clean results.

$\endgroup$
1
  • $\begingroup$ That's brilliant! Thanks @EdM for your help with this! $\endgroup$
    – docBen
    Aug 24, 2022 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.