The p-value comes from comparing how unusual your observed effect is relative to the false placebo effects (the ones where you run synth on untreated units). If it is unusual, you can reject the null of no effect.
You can use any measure of effect size that is comparable across units, but Abadie et al. (2010) suggest using post-treatment RMSPE (root mean square prediction error). That is the numerator of the ratio. The idea is that if there is an effect, the RMSPE will be large. So will be the MSPE. So if MSPE is 100, RMSPE is 10. If MSPE is 25, then RSMPE is 5. It does not really matter which you use.
So where does the denominator come from? The reason for the normalization by the pre RMSPE is that the placebo effects may be quite large if those units where not matched well in the pre-treatment period. This would cause p-values to be too conservative (i.e., you would not reject often enough). Adjusting for the quality of the pre-treatment matches can be done in two ways:
- Restricting the comparison set of placebos to only include those that match well, throwing away FPs with pre RMSPE X times greater than treated. Abadie et al. use X=5 in the Basque terrorism example.
- Dividing all post RMSPE by the corresponding pre-treatment match quality (RSMPE) to standardize them.
The second is simpler since it does not entail picking X in an ad hoc manner.
In the end, the two-sided p-value is the proportion of false placebo ratios that are greater or equal to the treated ratio. Some people will do that calculation with the high pre RMSPE placebos also thrown out. You can convince yourself of this by running a FP loop and storing the ratios along the way and doing the proportion calculation at the end.