I'd like to assess the impact of an upcoming policy implementation, as measured by changes in questionnaire response to a Likert-scale question.
I understand I could use a difference-in-difference approach. However, in my situation there is no single obvious comparison, non-treated population. I think I'd like to use the "Synthetic Control Method for Comparative Case Studies" as described by Abadie et al and implemented as Synth in R.
- Alberto Abadie, Alexis Diamond, Jens Hainmueller. Journal of the American Statistical Association. June 1, 2010, 105(490): 493-505. doi:10.1198/jasa.2009.ap08746. full text
As summarized in the R help for synth:
synth estimates the effect of an intervention of interest by comparing the evolution of an aggregate outcome for a unit affected by the intervention to the evolution of the same aggregate outcome for a synthetic control group.
synth constructs this synthetic control group by searching for a weighted combination of control units chosen to approximate the unit affected by the intervention in terms of the outcome predictors. The evolution of the outcome for the resulting synthetic control group is an estimate of the counterfactual of what would have been observed for the affected unit in the absence of the intervention. [..] the synth function routinely searches for the set of weights that generate the best fitting convex combination of the control units. In other words, the predictor weight matrix V is chosen among all positive definite diagonal matrices such that MSPE is minimized for the pre-intervention period.
See also the useful summary by Srikant Vadali in answers below.
Is this method appropriate for survey/sampled data? Is there anything I need to do differently, or just use my Likert-response mean as the dependent variable? Any suggestions about how I'd power such a beast?