I am interested in analyzing the effect of a specific change in traffic conditions on the amount of road accidents in a city.

I need to select a comparison group; my approach is to select from possible candidates, the city\district that was in the past most similar to the city being investigated .

Monthly data is available to me for all the cities.

However, I do not know if I should take as the comparison group the city most similar to mine comparing across the months, or rather sum the months, and use the city most similar to mine judging by the years.

I do not have access to a transportation expert whom I could ask for advice that would be relevant to the specific situation.

Are there statistical considerations to make the choice?

-----Added for clarification----

The way I judge the similarity between two cities:

For each city in the control group, there is a monthly amount of accidents, C(t). For the treatment group, I will denote the amount M(t). If the treatment is similar to the control group, then in the period before the treatment M(t+1)/M(t) = C(t+1) / C(t).

Which means that the rate of change is the same between the control and the treatment group. As far as I understand, this assumption allows me to use difference -in-differences to test for the effect of the treatment.

This is based on the procedure descirbed in the book "Observational Before-after Studies in Road Safety" \ Ezra Hauer.

However, he doesn't address there the question whether I should work with the monthly data, or sum it and select according to the yearly data.

  • 1
    $\begingroup$ Is it not the case that a city closest to your city of interest by months will also be closest by years? Using months is clearly a more stringent requirement, which, if satisfied, would yield a more valid effect estimate, but it may be harder to find such a city. This is the classic issue of the bias-variance trade-off in observational studies. $\endgroup$ – Noah Jul 26 '18 at 17:48
  • $\begingroup$ Thanks @Noah , you nailed it, it is indeed an issue of bias-variance trade-off. Precisely for that reason,using months is a "more stringent requirement", only if by stringent you mean having less bias. According to the trade-off, using months should result in less bias, but in more variance. Using years should result in more bias, but less variance. This is precisely what I see in the past data. (Btw, according to the data, the control group closest in term of years, is indeed not the closest one in terms of months.) I will present both, or will look at the synthetic group approach suggested. $\endgroup$ – Sam Jul 28 '18 at 9:02

Look up the synthetic control method in Abadie, Diamond & Hainmueller (2010). It is purpose built for studying problems like yours.

A specific policy change was made in city A, you want to know its effect. You have several candidate cities that you have longitudinal data for. The synthetic control method creates a control group whose weighted outcome matches the outcome for city A prior to the intervention. Then the intervention effect arises if city A diverges from the synthetic control post intervention.

It's a data driven method for finding the control group, but the weights are visible to you. And it is implemented in Stata and R.

  • $\begingroup$ Even if I use a synthetic control group, the question I have raised still remains. Should I construct a synthetic control group, (and make predictions) when I work with data from each month? Or should I sum the months, and construct a synthetic group across the years ? Thank you $\endgroup$ – Sam Jul 27 '18 at 15:43
  • 2
    $\begingroup$ @SamR. If you read the paper, you will find that the software finds the synthetic control for you, shows you how well it matched the treatment in the past and the divergence in the future. There are additional checks you can make to help your argument about causality. See jstatsoft.org/article/view/v042i13 for the JStatSoft paper. $\endgroup$ – Heteroskedastic Jim Jul 27 '18 at 18:37
  • $\begingroup$ Thanks @user162986. I run into a difficulty attempting to use this method. If you could help me with that, it would be great. stats.stackexchange.com/questions/364675/… $\endgroup$ – Sam Aug 31 '18 at 8:07

In my opinion, the matter is complicated and cannot be answered by a two-line note. However, for "treatment effect" estimation you may use a "propensity score matching" for control group selection (other approaches exist as well). See e.g., here or here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.