What statistical methods are used for estimating the effect of political policies?

Question

I've started to become interested in the statistical analysis of the efficacy of political policies on, say, health or economical outcomes.

I have a background in epidemiology. Here, estimating the efficacy of some treatment is a matter of identifying the right confounders and then use a method like g-computation of TMLE. I was wondering if the same reasoning can be used when analyzing policies? Or are there considerations here that merit a different type of analysis? Moreover, what is the current 'bible' about statistical methods for policy analysis, if there is any?

Answer 1

Pretty much every tool in the whole statistics toolbox can be used to evaluate policy. The real problems (compared to epidemiology) are often more in the design and the data:

• It's often difficult, impossible, unethical, or ridiculously difficult to randomize
• Many of the variables are measured with a lot of noise
• Many of the variables are only somewhat loose proxies for theoretical variables (although, from the question, you may not be interested in the theories)
• The models are often less powerful and/or more complex (in terms of number of variables) than those in epidemiology.
• The effects (if there are any) often take a long time to show up.

The above is a bit exaggerated to make a point -- I know that some epi models are very complex and that not all effects in epi are large and so on. But, in general, having worked with both epidemiologists and public policy people, I think they hold.