Best method for paired survival/time-to-event non-inferiority sample size calculation (using median survival time)?

Question

I am trying to design a theoretical study in which one group of patients uses two wearable devices (i.e., every participant uses both devices) that gather information about their vitals, which is then fed into an algorithm that predicts their risk of heart failure. I am thinking of using a time-to-event analysis, in which the time to event is the time between the point at which the algorithm predicts a heart failure event, and then the point at which the patient is hospitalized for heart failure. (e.g., the algorithm may predict a heart failure event on Day 1, but the patient won't be hospitalized until 4 days later. The longer this time is, the better, since I am trying to identify the device with the earliest prediction.) I want to compare the performance of the two devices (and their respective algorithms) in predicting heart failure events, meaning that the data will be paired, since each participant is providing data for both devices. Since I already know the median prediction time for one of the devices is 6.5 days, based on a prior study which inspired the current design (Free access: https://www.ahajournals.org/doi/10.1161/CIRCHEARTFAILURE.119.006513), I want to use a non-inferiority approach to demonstrate that the other device is at least as effective as the former.

Since this is all a theoretical study, the most important thing for me at the moment is to be able to identify the sample size that would lead to results with high power. (I also don't know if stratifying my population would affect things, but I was also hoping to do that.) My problem at the moment is that the study linked above doesn't have any information on hazard ratios, so I'm not sure how to proceed with median survival time as my reference.

What technique would be best for something like this? As you probably can tell, I'm not a statistician, so if software tools exist that can do something like this, I would prefer to use those.

Sorry for the huge question, and thank you in advance for your help!

Answer 1

After looking at the linked paper, I think that you will be best off with simulation to estimate sample size and power, based on the details of the clinical situation, the way you will estimate "non-inferiority," and the non-inferiority margin you pre-specify as acceptable. That's often the case in complex situations like this. Quoting from the paper, starting with 100 participants:

There were 49 hospitalizations that took place during the 90 days of follow-up in 38 subjects at a median time from discharge to rehospitalization of 50.5 days. Of these, 27 were HF [heart failure] hospitalizations and 40 were unplanned nontrauma hospitalizations (Figure 3). Sensor use compliance failure by the subject led to 5 events with insufficient data. These 5 events were excluded from the ROC analysis but are included in the time-to-event analyses.

Fifty-two emergency department visits took place during the 90 days of follow-up, of which 28 resulted in hospitalization...

Twelve subjects died during the study. Six deaths were adjudicated as sudden cardiac death, 2 were due to stroke, one due to HF, one due to sepsis, and in 2 subjects cause of death could not be determined. None of the sudden cardiac deaths were preceded by an alert in the 10-day positive window. (Emphasis added.)

That's a lot of complicated competing risks to evaluate, so a simple power analysis is probably unrealistic. Furthermore, at an initial reading I didn't see that the paper put together the prediction probability and the time between the warning "alert" and hospitalization very well. The median "alert-to-hospitalization time" in Table 2 was expressed only for cases where an alert happened in an appropriate "window" prior to the hospitalization; it seems to have ignored the false-positive "alert" events and the false-negative lack of "alert" prior to hospitalization. So you also need to consider what is the best way to compare the two paired monitoring strategies fairly, considering not only the median alert-to-failure time but also the failure-to-alert and false-alert probabilities.

To simulate data, you will have to devote a good deal of thought to the various types of events and their associated probabilities over time, and how the "alerts" of both types might be distributed in time, both with and without subsequent hospitalizations. That's hard, but might be a good thing as it forces you to think through what's going on with respect to both clinical status and how best to evaluate performance of the "alert" alternatives.

I very strongly recommend that you work closely with an experienced statistician on this, as it's a good deal more complicated than a simple time-to-event survival analysis. If this is ever to be used clinically with FDA approval you will need to work closely with a statistician anyway, so you might as well start now.

Finally, although you talk of using "non-inferiority" testing here, the way you evaluate "non-inferiority" for a new prediction model might be more complicated than in a non-inferiority study comparing a new therapy against an established standard of care. See this page for some elaboration.