Synthetic Control Method: Why average pre-treatment controls?

Question

The help file of the `synth' package in Stata states that they average the value of each matching variables across the whole pre-treatment period. Why not match on the value of X in each pre-treatment period?

Answer 1

I asked myself the same question when studying SC. The averaging of covariates X is not motivated by modelling, but by technical complexity.

In traditional Synthetic Control, we have to find weights $\mathbf{W}^*=\left(w_1^*, \ldots, w_{N-1}^*\right)^{\prime}$ that minimise the Euclidean distance between the predictors of treated and untreated units in the pre-treatment periods, subject to the weights being positive and summing up to 1.

Formally, \begin{aligned} &\min _W\left\|\mathbf{X}_1-\mathbf{X}_0 \boldsymbol{W}\right\|=\sqrt{\left(\mathbf{X}_1-\mathbf{X}_0 \mathbf{W}\right)^{\prime} \mathbf{V}\left(\mathbf{X}_1-\mathbf{X}_0 \mathbf{W}\right)}\\ &=\left(\sum_{k=1}^K v_k\left(X_{k N}-w_1 X_{k 1}-\cdots-w_{N-1} X_{k N-1}\right)^2\right)^{1 / 2}\\ &\text { where } \mathcal{W}=\left\{\left(w_1, \ldots, w_{N-1}\right)^{\prime} \text { subject to } \quad w_1+\cdots+w_{N-1}=1,\right.\\ &w_i \geq 0 \text {, }\\ &\text { with } i=1, \ldots, N .\} \end{aligned}

and where:

• $i=1,...N$ is the unit index, $i=\{N\}$ is the only treated unit, while $i=\{N-1\}$ are the control (or donor) units;
• $k=1,...K$ indexes covariates; and
• $t$ indexes time;
• $w_1,...w_{N-1}$ are the weights that, once optimised, are used to build the synthetic control unit.

If we did not take the mean or median or some other statistic of the covariates, the problem (if formulated this way) would not be solvable as $\mathbf{X_1}$ and $\mathbf{X_0}$ would no longer be matrices, but 3D objects (tensors).

Long response to the comment:

In the problem statement above, $X_{k N}-w_1 X_{k 1}-\cdots-w_{N-1} X_{k N-1}$, the covariates change per each unit, $i=1,...N$, but not over time. Notice also that the averages you mentioned in your question are one "long average" per each unit. E.g. you follow the GDP of a region over 10 years and take the average of those values, and then you do this for each donor region in your dataset.

Your dataset should look like a matrix where each row is a unit-period (each unit has multiple rows, one per time period) and each column is a single variable, changing over time as you go down the column.

If you make one column per variable per year per unit, you end up with $N \times T$ columns with only one non-missing element each. Your variables have become constants and hence useless for statistical purposes.

If you make one column per variable per year, the data is not shaped in the correct way and your statistical software is misinterpreting your input and giving you the wrong results. In practice, it is still taking averages, but the wrong ones. It is taking one average per year (across units) instead of one average per unit (across years).

I hope this helps!

Answer 2

Complementary to Marti's answer (+1): when we have multiple matching variables it becomes increasingly harder to find a single match/near-neighbour that is satisfactory. By "averaging" we get a synthetic match that is optimal based on our selected metric. Of course, if we could find a control instance that directly matches our treated instance perfectly in all the matching pre-treatment variables we should use it! :)

Answer 3

The main reason is to 1) prevent overfitting and 2) ease interpretation.

To understand 1), compare two cases; in one case we only use pre-period averages and in another case we use every individual time period. Both cases have very good pre-treatment fit of the dependent variable of interest between the synthetic control and our treated unit. We will be much more trusting in our synthetic control that uses averages because then we have essentially shown that there is co-movement in the outcome of interest across the control units and treated unit, without directly targeting this co-movement to fit on. If we included all time periods, we are ex-ante forcing the model to choose a synthetic control that fit all of the pre-period well, i.e., we at risk of overfitting. That doesn't mean that by fitting all time periods always overfits, but it could be and thus isn't very credible.

For 2), note that subject context is extremely important when doing synthetic controls. With enough varied controls we can get a perfect pre-treatment fit but that synthetic control would be garbage. The only time we can use synthetic controls is if we have a case where we don't have a perfect control but rather have a pool of similar non-treated units to compare to. For a credible synthetic control exercise we need to show that the weights/controls make sense logically. The clarity of the exercise is stronger when we have fewer measures. Synthetic controls is not magic, we need to really believe in the validity of the synthetic control we create.

There is absolutely no mathematical reason why you could not include every period instead of taking the average! We can treat each time period as a completely separate piece of data (i.e., then the canned code would take a mean over a singleton). For this reason I disagree with Marti's answer.