I am running difference in differences analysis. To show that results are not driven purely by the research design, I want to run placebo analysis. However the issue with my setting is that, each individual receives treatment at different time periods and they can receive treatment couple of times. My baseline regression is this: Yit = α + β1 × Postit + γi + δt + ϵit, where γi is individual and δt is time fixed-effect. What would be the best way to run placebo analysis? I think i can not use lead values as they might receive additional treatment later on. Or should I simply run coarsened exact matching?

  • $\begingroup$ I am assuming that your protocol says that the timing of placebo administration is the same as active treatment. If not, it is not a placebo. If the treatment is an injection, the control must be injected with saline. Matching a placebo is necessary for invasive medical routes that involve pain, injections, long travel, or personal questions. $\endgroup$ – AdamO Mar 26 '18 at 18:56
  • $\begingroup$ What I mean by placebo is actually falsification test $\endgroup$ – edyvedy13 Mar 26 '18 at 19:09
  • $\begingroup$ I think you should expand your question to explain what you're doing, in whom, and why to get more useful answers. There's a lot of confusing use of specific terminology: a placebo is a simulated treatment. $\endgroup$ – AdamO Mar 27 '18 at 14:39

You specification doesn't handle multiple treatments, so this false placebo test follows a similar approach.

1) Drop all the outcomes for treated observations after they receive treatment for the the first time. Everyone in the remaining data should only have untreated outcome data.

2) Insert a phantom treatment event in the middle of the remaining data for the treated group. You might have to break some ties if you have an even number of periods.

3) Run your diff-in-diff model and check the interaction coefficient.

  • $\begingroup$ Nice approach. I agree the baseline status of treatment should be used somehow, although my answer doesn't do that. I think "placebo" effect being an effect of being "on treatment" requires some thinking about how to code compliance. See my discussion about per protocol vs intent to treat analyses. Also I asked a relevant question here that I'm curious if you have any input. $\endgroup$ – AdamO Mar 26 '18 at 19:04
  • $\begingroup$ Thank you very much for the answer, do you know any paper that I could give referance to for this type of placebo analysis? $\endgroup$ – edyvedy13 Mar 26 '18 at 19:12
  • $\begingroup$ Another limitation I might make mention of is prevalent case bias and attrition bias. Rarely does status remain constant over the course of a trial, so status follows a trend. Attrition bias is a tendency to discontinue treatment because it performs poorly. Together, these mean that those in early phases are more likely to adhere. And those in later phases are less likely to adhere if their status is worse. I worry that an unmatched analysis (even using LOCF which I think you call phantom treatment) is not going to address this. $\endgroup$ – AdamO Mar 26 '18 at 19:28
  • $\begingroup$ @edyvedy13 Unfortunately, I am not aware of any problems that do this formally. People do something like this in the synthetic cohort literature, but it is for the purpose of calculating p-values, rather than testing the common trends assumption. $\endgroup$ – Dimitriy V. Masterov Mar 26 '18 at 19:30
  • $\begingroup$ @Dimitriy V. Masterov, thank you very much in any case it makes sense to me let's see what reviewers will say. @ AdamO You are right, I should match observations in treatment and control in any case $\endgroup$ – edyvedy13 Mar 26 '18 at 19:38

There is no clear answer: but I can offer two with the notion that they address two different questions.

We all know compliance is a function of status. People who perform poorly are less likely to receive treatment on time, or at all. Some terms trialists use to describe the treatment effects are "effectiveness/efficacy" or "use/method efficacy". The analysis plan description is an "intent to treat" (ITT) analysis: meaning "I said I would give that patient placebo, and whatever the heck they did after I randomized them to it is representative of people who get placebo." (I hope it's a blinded study).

One approach is to analyze the imbalanced data using a mixed model, the timing of the treatment is irrelevant. Some research shows this model has a tendency to "project out" outcomes. This is adequate for an "intent to treat" analysis. For binary events, when timing of treatment is known, and a suitable dropout can be defined, a survival model can be fit with time varying covariates. In both cases, "treatment" is a constant term (for ITT) because of the reasoning I mentioned earlier.

ITT's converse is a per protocol (PP) analysis. Here we say, "What would have happened if people did as I told them to?" The problem is that in this case there are really three treatments: active, placebo, and off-treatment (or non-compliant). Some serious thought has to be put into the pharmacokinetics of treatment. For instance, if I receive dialysis on Monday, after what time am I no longer "on dialysis"? Do you code me as 1 for 5 days, or a tapering-time exposure of 1 on day 1, 0.8 on day 2, 0.6 on day 3, etc. This is a common analysis approach for phase 2 trials. Time varying covariates are a huge subject of debate and cause a lot of confusion.

PP analyses can make use of the previous mixed and survival models to predict survival conditional upon status and treatment. The survival curve does this vis-a-vis the baseline hazard function. The mixed model can even be taken one step further to predict later outcomes (dropouts tend to have random intercepts and random slopes depicting their rate of descent in status and likelihood of attrition). If dropouts are "aliased" so that suitable predictions of status (and treatment) cannot be projected over the trial duration, more sophisticated models can be applied: the outcome can be imputed with last observation carried forward, worst observation carried forward, or complex Markov models.

Testing for a placebo effect can be done with the same analytic approach. Using only those randomized to placebo, use the time varying covariate of receipt of placebo, and perform the T-test. I would not do difference-in-differences with treated because active treatment often has cumulative effects. Their receipt of treatment does not make them generalize to the "untreated" patients except with baseline (pre-randomization) measures.

https://www.rti.org/sites/default/files/resources/mr-0009-0904-chakraborty.pdf http://www.nejm.org/doi/pdf/10.1056/NEJMsr1203730

  • $\begingroup$ Sorry for the misunderstanding, actually my research designs is not closely related to medicine. What i was trying to do was doing something similar to checking parallel trends assumption $\endgroup$ – edyvedy13 Mar 26 '18 at 19:12
  • $\begingroup$ @edyvedy13 hmm a difference-in-difference test will not tell you that. $\endgroup$ – AdamO Mar 27 '18 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.