Treatment evaluation: Measuring statistically / clinically significant change in level over time

Question

I am conducting a treatment evaluation. I am using an interrupted time series design (generalised linear mixed model; 42 monthly measurements per patient [18 pre-treatment, 24 post-treatment). I have assessed the impact of the treatment program on the level and trend of an adverse outcome for the whole sample (treatment vs no treatment) and would now like to assess whether the impact of treatment on the outcome is moderated by a statistically / clinically significant change in the provision of services. That is, is the effect of treatment "conditioned" on an increase in contact with health services.

Initially, I considered calculating change in contact using the reliable change index (e.g., baseline, mean amount of contact in X months before treatment; post-treatment, mean amount of contact in X months following treatment); however, I would really like to calculate change from the baseline level over time to the post-treatment level over time.

I have also considered comparing the baseline and post-treatment levels using another interrupted time series model, but my understanding is that the estimate for the post-treatment level will be for the time immediately following implementation of the treatment program and I am unclear how to extrapolate this to a level over time.

Question

What is an appropriate way to calculate statistically / clinically significant change in the level of a variable using multiple observations?

Answer 1

This does not directly get at your question, but since you seem to be ultimately interested in controlling for this change in your main ITS model, have you considered applying a time-varying covariate approach by interacting monthly amount of contact with your "time since the start of the study" variable, perhaps only in the post-intervention period (if you only found an effect on the slope in the main model)? Then you will be able to test the significance of those variables and would not have to worry about how you've specified the change.

So, if the original ITS model (without any covariates or random effects) took the form of:

$$ Y_{t} = \beta _{0} + \beta _{1}T_{t} + \beta _{2}X_{t} + \beta _{3}X_{t}T_{t} + \epsilon _{t} $$

where $Y_{t}$ was monthly level of adverse outcome, $T_{t}$ was the time since the start of the study, $X_{t}$ was a dummy representing pre- and post-intervention periods, and $X_{t}T_{t}$ was the interaction term, the modified model that tested whether change in contact moderated the values of $\beta _{2}$ (change in the level of the outcome) and $\beta _{3}$ (difference in slopes of the outcome) would be this:

$$ Y_{t} = \beta _{0} + \beta _{1}T_{t} + \beta _{2}X_{t} + \beta _{3}X_{t}T_{t} + \beta _{4}X_{t}C_{t} + \beta _{5}X_{t}C_{t}T_{t} + \epsilon _{t} $$

where $C_{t}$ measured monthly amount of contact, $\beta _{4}$ tested the difference in pre- and post-intervention periods by average level of contact in the two periods, and $\beta _{5}$ tested if the difference in slopes varied by level of contact.

If you wanted, you could define $C_{t}$ not as monthly amount of contact but as monthly change in contact (in absolute or relative terms).