5
$\begingroup$

Was hoping to get your thoughts on a prospective study design. Here's some basic info:

Population: Patients with melanoma receiving immunotherapy. Exposure: Steroids within 3 months prior to first dose of immunotherapy Outcome: Overall survival

Hypothesis: Exposure to steroids within 3 months prior to the first dose of immunotherapy decreases the efficacy of immunotherapy through suppression of the immune system.

We have retrospective data suggesting worse survival for patients who received steroids, even when controlling for the indication for the steroids. Therefore it's not ethical to randomize patients to steroids, so I was thinking the next best thing would be a prospective cohort study. We'd enroll patients getting immunotherapy for melanoma and record who received steroids within the 3 months prior to the first infusion. We'd then follow them forward in time and derive a relative risk for steroid exposure.

Any other thoughts on what an optimal study design would be to prove harm from steroids?

Thanks!

$\endgroup$
3
  • 3
    $\begingroup$ Not to dissuade you from seeking advice here, but I'd strongly recommend that you consult/collaborate with an expert. There are clear medical and ethical consequences to your question, and I wouldn't be comfortable proposing a solution based on reading a 150-word description. The details are likely to matter, and we are probably insufficiently informed to give you the best answer. $\endgroup$
    – mkt
    Aug 6, 2019 at 15:19
  • 1
    $\begingroup$ I absolutely will be consulting with a statistician here, but that's down the line and I was just curious to hear what methodologic options there are to be a better informed investigator. If it makes you uncomfortable you can ignore all the medical aspects in my post and just focus on the hypothesis: Exposure A reduces the efficacy of drug B. My question is what is the best study design to support the hypothesis. $\endgroup$
    – JJM
    Aug 6, 2019 at 18:43
  • $\begingroup$ It certainly helps to know that this question will be followed up with a consultation. I'll give your question more thought. $\endgroup$
    – mkt
    Aug 7, 2019 at 6:40

3 Answers 3

3
$\begingroup$

I start with my methodological thoughts and I offer some footnotes with thoughts that came to my mind on the ethics. Take both of these with a large grain of salt, because we know very little on your specific case. That makes both methodological advice, as well as comments on the ethics difficult and my remarks could completely miss key considerations.

Randomization (either to steroids vs. alternative treatment*, or with respect to delaying immunotherapy**) in theory is your best bet for really establishing causation. If you truly cannot do that, then think about why a prospective study is needed; one reason might be that it's easier to get information on all possible confounders and exposures than in a retrospective design, if so, make sure that you really get this information. Just being able to write "in this prospective study..." in a publication is usually not considered an adequate reason for a prospective study. Alternatives to prospective studies include, for example, retrospective cohorts or case-control studies.

If you do not randomize, you will end up somehow matching patients (either into small groups or strata) or adjusting for confounders in some manner. You may run into some serious difficulties here.

Firstly, it it's about steroids or not, then it might simply be the case that the medical conditions that required steroid treatment lead to a worse prognosis and if nearly all patients with these medical conditions (or most of the ones with the worst severity of the condition) get steroids, there might be no realistic way of adjusting this away, or finding truly matched patients with the same (or equally severe) history that did get and did not get steroids. It might also be the other way around: the worse/the more life-threatening the melanoma, the more prone patients might be to get conditions that require steroids (e.g. due to the melanoma or due to previous treatments for them). Thus, one big question is whether there are alternative treatments instead of steroids that are used for the conditions for which the steroids are used. If there are and if the choice of treatment is based on somewhat random physician preferences, then that's the best situation for a non-randomized study. If there are not, you will have a really hard time (=it may not be possible) disentangling things.

Secondly, when one looks at the time of initiation of immunotherapy (if you have the theory that the longer ago the steroids were used the better), then "longer ago" vs. "more recently" might still be a serious confounding factor (e.g. "more recently" might mean that the patient has not fully recovered and this affects their prognosis) just like what is discussed in the previous paragraph.

* It is always a difficult judgement whether randomization is ethical. In part, whether giving steroids is still ethical will depend on the strength of evidence that is already available. However, if you believe you can still find patients that would get steroids for a prospective study, it seems there is disagreement on this and/or there might be a population where the benefit/risk is still considered acceptable by some physicians. This obviously needs careful consideration, but usually pretty compelling data is needed to truly change clinical practice. Additionally, sometimes the problem might be the other way around - there might be patients where withholding steroids is not ethical.

** Delaying a potentially life-saving cancer therapy in a randomized fashion of course has serious ethical implications, too. Thus, this may not be a good target for intervention, either. The timing could be something to look at in an observational fashion, because one could then have patients with a more similar medical history. However, see second caveat.

$\endgroup$
1
  • $\begingroup$ Thanks for your thorough reply. You gave me a lot to think about and raised some really important issues. This will really enhance my conversation with our statistician when I meet with them. All of your responses were very helpful! $\endgroup$
    – JJM
    Aug 15, 2019 at 14:05
3
$\begingroup$

We have retrospective data suggesting worse survival for patients who received steroids, even when controlling for the indication for the steroids. Therefore it's not ethical to randomize patients to steroids,

This is exactly why you have to do randomized controlled trials. The result is not surprising: someone who needs steroids is going to be much sicker than someone who doesn't. It's prevalent case bias.

To the best of my knowledge, immunotherapies don't interact with steroids, but rather people on immunotherapy have intermittent neutropenia and are more prone to infection, and it becomes increasingly challenging to differentiate actual infection from the treatment effects of Rituximab, etc. However, if oncologists are not diligent in pathology, their patients are treated for inflammatory conditions using what would be the standard of care in most cases. Famous example is: lung infection (pneumonia) vs non-infectious pneumonitis.

The best solution is to

  1. Identify standard of care, and revise protocol to make sure that appropriate diagnostic steps are made regarding treatment. Require investigators to report pathology data, and other hemology like WBC, cRP, etc.
  2. In situations where investigators have inconclusive pathology, perform a randomized design to use steroidal vs. non-steroidal treatment.
  3. Follow for time to first worsening of adverse event, or death. Test for differences with log rank or $G \rho \gamma$ weighted survival
$\endgroup$
2
  • $\begingroup$ Thanks for your reply. This is a good way to do a RCT in an ethical way. Rather than randomizing people with soft indications for steroids to steroids as an experimental arm, you could consider non-steroidal treatment as the "experimental" arm, since you have some low quality evidence to suggest it may be less harmful. $\endgroup$
    – JJM
    Aug 15, 2019 at 14:04
  • $\begingroup$ Exactly. You have equipoise (at least as you describe it with me, but best to align with clinical care guidelines and provider perspectives). As with most trials, it's important to carefully identify the correct study population. As I mentioned, care mistakes abound and not only are they harmful to the patient, but reduce the chances of conducting a successful trial (which in turn affects the health of future patients). $\endgroup$
    – AdamO
    Aug 15, 2019 at 15:21
2
$\begingroup$

The scholarly literature guiding you into the best way to test a scientific hypothesis of comparative clinical effectiveness is quite large.

Have a look for instance at Rosenbaum's Design of Observational Studies: https://www.amazon.com/Design-Observational-Studies-Springer-Statistics-ebook/dp/B00DZ0PT76/.

Having said this, I think some informal guidance can indeed be provided. First, carefully decide what you want to test/measure and which patients you want to inform when the study is completed and the data collected. This will tell you which patients to include and which endpoints to focus on. Second, ensure best research practice (irrespective of the presence/lack of randomization): explicit selection criteria, formal data collection process, validated outcome ascertainment. Third, figure out the sample size: this may depend on pragmatic issues, funding, or a formal power analysis, but in general terms the more patients (from several centers), the better. Regarding analysis, there is plenty of options. Most experts would argue however that propensity matching, inverse probability of treatment weighting, or instrumental variable analysis represent the most advanced and less likely to be biased ones.

Bottomline: a prospective observational study is always a good start to confirm a promising retrospective one, but remember that association is not causation, and in the modern era of industrial medicine most treatments need to be supported by randomized trial data...

$\endgroup$
1
  • 1
    $\begingroup$ " and which patients you want to inform when the study is completed and the data collected." I like this way of thinking about selection criteria. Always a balance between being too inclusive vs. too exclusive. $\endgroup$
    – JJM
    Aug 15, 2019 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.