When running an RCT where different individuals are invited to participate and then randomized, what would the interpretation of diff-in-diff method be? To my understanding, diff-in-diff identifies the Average Treatment Effect but here we should be dealing with the Intention to Treat , right?
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Usually you calculate the average effect on the treated (ATT) in DiD, see this great resource, when treatment assignment is non-random and endogenous to some unobserved charateristics. (But DiD requires some strong assumptions to be able idenify ATT)
When you run an RCT, you can estimate ITT easily by a simple group comparison, because assignment is random. If you are interested in the local ATE and compliance/uptake is not 100% (this refers to 100% among those randomized, not 100% among those invited to participate in the first place), then you can use randomization as an instrumental variable, as mentioned in this thread.
Consequently, I don't see how you would possibly like to combine these methods, as they apply to different study designs. Rather, if many people decline the invitation to be randomized/included in the study, you might want to think about what the population finally included in your RCT actually represents. If that population is different from the target population you want to study regarding some observables, you can use poststratification after effect estimation, as is well explained in this paper by Stuart et al. (2015).
$\begingroup$
$\endgroup$
DiD is aplicable for RCT but note that it’s a causal inference method originally developed for contexts where RCT isn’t possible.
Fundamentally, DiD accepts that there is imbalance at the start of an observation period (pre-start period in RCT context) but crucially that the treatment group mean and control group mean are increasing/decreasing at the same, uniform weight. This is the basis of the counterfactual. (What would the treatment group have done in the absence of treatment exposure?)
If you’re executing an RCT and have the ability to randomize, you might be able to eliminate imbalance between groups to some arbitrary, negligible level. But for DiD to be helpful, you need to ensure that both group means are changing at negligible different rates (over time.)
Taking change in mean over time into account when randomizing can help ensure that you meet theoretical requirements necessary to leverage DiD.
Remember, you won’t observe how the Treatment group mean changes over time in the absence of treatment exposure, this is only observed for the control group group. So DiD assumes that the treatment group mean changes at the observed control group rate.
By including change in mean over time in randomization, DiD could be very a good fit for you even if imbalance between group means is pronounced.
