2
$\begingroup$

I'm attempting a 'difference-in-differences' analysis of a health policy intervention, over 3 years. I'd appreciate advice on my methodology.

Scenario:

Year 0: Health clinics are given financial incentives for offering three different treatment indicators (e.g. for diabetes, for high blood pressure, for asthma).

Year 1: Control group (asthma) continues with incentives, as in year 0. The two other indicators (the two intervention groups) stop being incentivised financially. (So I can assess the impact of dropping the financial incentives with a DiD approach, after 1 year.)

Year 2: The control group continues as before. Intervention 1 (diabetes) goes BACK to being incentivised (as in year 0). Intervention 2 (high blood pressure) continues without being incentivised financially - but clinics are told they should use 'peer review schemes' to maintain performance.

Is there an effective way to quantify the impact of:

a) intervention group 1 returning to the original payment incentive in year 2

b) intervention group 2 continuing without payment incentives in year 2 (as in year 1) - albeit with new requirement that they maintain performance with peer review.

For each year (0, 1, 2) I have data for each of the three groups (control group, intervention group 1, intervention 2) for every single clinic in the country (around 450 clinics).

My approach so far:

  • Do a standard DiD comparison between years 0 and 1, to assess impact of stopping financial incentives (comparing control and intervention groups)

  • Do an extended DiD analysis of intervention 2, to assess impact of stopping financial incentives over two years - comparing it to the control group. This approach isn't perfect because, after year 1, clinics were told explicitly to maintain intervention 2 performance with peer review (so the situation changed slightly). But I'd also like to draw conclusions about whether or not performance was maintained after the introduction of 'peer review' (presumably that can be done without a DiD analysis)

  • Do a DiD analysis of intervention 1, from year 1 to 2 (to assess impact of returning to financial payments, after dropping them in year 1) - use intervention 2 as the control group (which continued without incentives from years 1 to 2). Again, this isn't perfect, because of the added requirement for intervention 2 (after year 1) that clinics maintain their performance with peer review.

Is there any way this approach can be improved? Thanks.

$\endgroup$
0
$\begingroup$

Do a standard DiD comparison between years 0 and 1, to assess impact of stopping financial incentives (comparing control and intervention groups)

You could create a model with multiple (two, in your case) treatment groups to assess heterogeneous treatment effects on the two intervention groups (see, e.g., Diff-in-Diff with multiple treatment groups).

Do an extended DiD analysis of intervention 2, to assess impact of stopping financial incentives over two years - comparing it to the control group. This approach isn't perfect because, after year 1, clinics were told explicitly to maintain intervention 2 performance with peer review (so the situation changed slightly)

For assessing the impact of changes in years 1 and 2 on intervention group 2, you could estimate one model with the entire two-year period used as one multiple-period post-intervention period and with indicator variables included for each of the two years which you would interact with the indicator for Treatment/Treated (see, e.g., Difference in Difference with multiple period (pre, during, and post treatment)). This way you could assess if there were differences in outcomes between years 1 and 2.

Do a DiD analysis of intervention 1, from year 1 to 2 (to assess impact of returning to financial payments, after dropping them in year 1) - use intervention 2 as the control group (which continued without incentives from years 1 to 2). Again, this isn't perfect, because of the added requirement for intervention 2 (after year 1) that clinics maintain their performance with peer review.

Same suggestion applies here as above. You could use the Asthma group as your control group here and just specify your model with multiple periods (during treatment and post treatment).

Also, this does not directly get at your questions, but it seems like in this type of policy analysis, time-varying treatment effects may be an issue if these incentive programs are announced prior to the start of each year and if the effects of policy change increase or dissipate over time. There are ways to model these anticipation and phase-in effects (see, e.g., https://www.annualreviews.org/doi/pdf/10.1146/annurev-publhealth-040617-013507).

Final thought: I hope you don't have overlaps between your groups. I suppose it is possible for a diabetes patient to also have high blood pressure.

$\endgroup$
  • $\begingroup$ Thanks v helpful. Re policy effect increasing or dissipating over time: in this case, longer-term studies on the same policy (the QoF) didn't show this - where the incentive was in place for several years, performance increased quickly and then plateaued (until you removed the incentive). Regarding overlap between groups: true, multi-morbidity is prevalent, so there will be overlap - however, my actual study will involve over 30 indicators/illnesses (I simplified here) - so hopefully I'll get a more accurate picture than if I just looked at three indicators. Many thanks for the reference. $\endgroup$ – hassapikos Apr 17 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.