0
$\begingroup$

my friends at Stackexchange, I have been contemplating and searching the literature for the best options to probe the interaction effect between treatment integrity (or fidelity) and treatment effect in RCTs. Let's focus on the simplest "gold standard" RCT = pretest-posttest control group design first. (fidelity = the degree to which a treatment is implemented as intended)

Problem parameters:

  • in my field, many RCTs do not collect fidelity data from the blank/placebo/business-as-usual control arm (understandable because nothing is done in control).
  • Even if fidelity data is collected from the control arm, it is NOT the same measure/metric as the one for the treatment arm.
  • But it is often reasonable to assume an interaction between fidelity and treatment effect in most cases.
  • Let's discuss this issue primarily assuming we use ANCOVA to analyze the data (i.e., DV= posttest, focal predictor= treatment, covariates= demographics + pretest + fidelity).

My two cents:

  • If we control fidelity as a covariate in ANCOVA, its coefficient (main effect) means the slope of fidelity in the control group only (b/c controlling for everything else as 0). Thus, it means nothing in our situation(constant 0)...

  • Well, let's ignore the nonsense main effect of fidelity and add an interaction term between fidelity and treatment. Then the coefficient of this term means the difference in slopes between the fidelity in the treatment group vs. control group.

  • since the control group has nothing about fidelity (constant 0), it basically means the slope of the fidelity in the treatment group.

My Qs:

I was wondering if our mighty SE community has better solutions, or identifies any issues in my humble two cents. Please do extend this discussion to other methods or designs (e.g., longitudinal RCT design)

Thank you so much for your time and efforts in advance!

Improve this question
$\endgroup$
4
  • $\begingroup$ In your question, does “fidelity” refer to staying on the assigned treatment? If so, I don’t see a way to separate the idea of “intention-to-treat” from “as treated.” However, you may want to search for those terms to better understand the implications of treatment “cross over.” $\endgroup$
    – Todd D
    Commented Jul 17, 2022 at 16:42
  • $\begingroup$ Thank you for your response! But ITT and principal strata are not relevant to this question. As mentioned at the fourth line: ...(fidelity = the degree to which a treatment is implemented as intended) $\endgroup$
    – YcZ
    Commented Jul 18, 2022 at 17:12
  • $\begingroup$ Your concept of fidelity likely falls under the concept of 'treatment adherence.' $\endgroup$
    – Todd D
    Commented Jul 19, 2022 at 17:02
  • $\begingroup$ Yes, fidelity has five dimensions, we usually focus on adherence in the field of psychology. :) Could you pls offer some ideas to my questions? Thank you! $\endgroup$
    – YcZ
    Commented Jul 19, 2022 at 22:42

1 Answer 1

Reset to default
0
$\begingroup$

Without equal measures of fidelity, I agree that adjusting for fidelity is infeasible. An interaction term does not help the situation.

This is an example of how the quality of a randomized trial is primarily determined by its design, not by statistical analysis.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for your response, Todd D! I do agree with you that design-based approach is better than mechanical ones. But this is the awkward situation many RCTs are facing in implementation or intervention science... Any suggestion for resolving this with study design modification? (do please read my "fine prints" as there are many caveats in my field ;) $\endgroup$
    – YcZ
    Commented Feb 27, 2023 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.