Study design question: What's the best design to assess harm of an exposure?

Question

Was hoping to get your thoughts on a prospective study design. Here's some basic info:

Population: Patients with melanoma receiving immunotherapy. Exposure: Steroids within 3 months prior to first dose of immunotherapy Outcome: Overall survival

Hypothesis: Exposure to steroids within 3 months prior to the first dose of immunotherapy decreases the efficacy of immunotherapy through suppression of the immune system.

We have retrospective data suggesting worse survival for patients who received steroids, even when controlling for the indication for the steroids. Therefore it's not ethical to randomize patients to steroids, so I was thinking the next best thing would be a prospective cohort study. We'd enroll patients getting immunotherapy for melanoma and record who received steroids within the 3 months prior to the first infusion. We'd then follow them forward in time and derive a relative risk for steroid exposure.

Any other thoughts on what an optimal study design would be to prove harm from steroids?

Thanks!

Answer 1

I start with my methodological thoughts and I offer some footnotes with thoughts that came to my mind on the ethics. Take both of these with a large grain of salt, because we know very little on your specific case. That makes both methodological advice, as well as comments on the ethics difficult and my remarks could completely miss key considerations.

Randomization (either to steroids vs. alternative treatment*, or with respect to delaying immunotherapy**) in theory is your best bet for really establishing causation. If you truly cannot do that, then think about why a prospective study is needed; one reason might be that it's easier to get information on all possible confounders and exposures than in a retrospective design, if so, make sure that you really get this information. Just being able to write "in this prospective study..." in a publication is usually not considered an adequate reason for a prospective study. Alternatives to prospective studies include, for example, retrospective cohorts or case-control studies.

If you do not randomize, you will end up somehow matching patients (either into small groups or strata) or adjusting for confounders in some manner. You may run into some serious difficulties here.

Firstly, it it's about steroids or not, then it might simply be the case that the medical conditions that required steroid treatment lead to a worse prognosis and if nearly all patients with these medical conditions (or most of the ones with the worst severity of the condition) get steroids, there might be no realistic way of adjusting this away, or finding truly matched patients with the same (or equally severe) history that did get and did not get steroids. It might also be the other way around: the worse/the more life-threatening the melanoma, the more prone patients might be to get conditions that require steroids (e.g. due to the melanoma or due to previous treatments for them). Thus, one big question is whether there are alternative treatments instead of steroids that are used for the conditions for which the steroids are used. If there are and if the choice of treatment is based on somewhat random physician preferences, then that's the best situation for a non-randomized study. If there are not, you will have a really hard time (=it may not be possible) disentangling things.

Secondly, when one looks at the time of initiation of immunotherapy (if you have the theory that the longer ago the steroids were used the better), then "longer ago" vs. "more recently" might still be a serious confounding factor (e.g. "more recently" might mean that the patient has not fully recovered and this affects their prognosis) just like what is discussed in the previous paragraph.

* It is always a difficult judgement whether randomization is ethical. In part, whether giving steroids is still ethical will depend on the strength of evidence that is already available. However, if you believe you can still find patients that would get steroids for a prospective study, it seems there is disagreement on this and/or there might be a population where the benefit/risk is still considered acceptable by some physicians. This obviously needs careful consideration, but usually pretty compelling data is needed to truly change clinical practice. Additionally, sometimes the problem might be the other way around - there might be patients where withholding steroids is not ethical.

** Delaying a potentially life-saving cancer therapy in a randomized fashion of course has serious ethical implications, too. Thus, this may not be a good target for intervention, either. The timing could be something to look at in an observational fashion, because one could then have patients with a more similar medical history. However, see second caveat.

Answer 2

The scholarly literature guiding you into the best way to test a scientific hypothesis of comparative clinical effectiveness is quite large.

Have a look for instance at Rosenbaum's Design of Observational Studies: https://www.amazon.com/Design-Observational-Studies-Springer-Statistics-ebook/dp/B00DZ0PT76/.

Having said this, I think some informal guidance can indeed be provided. First, carefully decide what you want to test/measure and which patients you want to inform when the study is completed and the data collected. This will tell you which patients to include and which endpoints to focus on. Second, ensure best research practice (irrespective of the presence/lack of randomization): explicit selection criteria, formal data collection process, validated outcome ascertainment. Third, figure out the sample size: this may depend on pragmatic issues, funding, or a formal power analysis, but in general terms the more patients (from several centers), the better. Regarding analysis, there is plenty of options. Most experts would argue however that propensity matching, inverse probability of treatment weighting, or instrumental variable analysis represent the most advanced and less likely to be biased ones.

Bottomline: a prospective observational study is always a good start to confirm a promising retrospective one, but remember that association is not causation, and in the modern era of industrial medicine most treatments need to be supported by randomized trial data...

Answer 3

We have retrospective data suggesting worse survival for patients who received steroids, even when controlling for the indication for the steroids. Therefore it's not ethical to randomize patients to steroids,

This is exactly why you have to do randomized controlled trials. The result is not surprising: someone who needs steroids is going to be much sicker than someone who doesn't. It's prevalent case bias.

To the best of my knowledge, immunotherapies don't interact with steroids, but rather people on immunotherapy have intermittent neutropenia and are more prone to infection, and it becomes increasingly challenging to differentiate actual infection from the treatment effects of Rituximab, etc. However, if oncologists are not diligent in pathology, their patients are treated for inflammatory conditions using what would be the standard of care in most cases. Famous example is: lung infection (pneumonia) vs non-infectious pneumonitis.

The best solution is to

1. Identify standard of care, and revise protocol to make sure that appropriate diagnostic steps are made regarding treatment. Require investigators to report pathology data, and other hemology like WBC, cRP, etc.
2. In situations where investigators have inconclusive pathology, perform a randomized design to use steroidal vs. non-steroidal treatment.
3. Follow for time to first worsening of adverse event, or death. Test for differences with log rank or $G \rho \gamma$ weighted survival