1
$\begingroup$

I have a question about control group selection.

I want to evaluate a private education suppression policy. The policy is that 70% of questions of a "national" university entrance test is given from the contents of public education class.

Here, I want to use the students who have long been keeping zero spendings on private education as a control group. (Since the university entrance test is common for all students in this country, we cannot utilize students in a specific area)

Is the construction of this control group problematic? Do you know any study that uses this kind of control group?

Thank you for your time spending to read this question.

$\endgroup$
  • $\begingroup$ Welcome. First, can you describe your outcome variable to us? Second, wouldn’t the students without any investment in private education also be impacted by the policy? Please provide further clarity here. Is there any reason why you can’t use students in a neighboring country? $\endgroup$ – Thomas Bilach Nov 9 '20 at 5:52
  • $\begingroup$ The outcome variable is the expenses on private education of a student. The second question is my question. In short, I am not confident to argue that the students without any investment in private education are not impacted by the private education suppression policy. So, I asked whether you guys think the assumption (students without the investment are not affected by the policy) sounds reasonable. I think, I have to consider about the possible use of other countries' students as a control group. $\endgroup$ – QWEQWE Nov 9 '20 at 8:26
0
$\begingroup$

If I understand your post properly, you are investigating the impact of an exam—administered at a national level to students—on their private education expenditures.

Since the university entrance test is common for all students in this country, we cannot utilize students in a specific area.

If you're using difference-in-differences, then it requires you observe a subset of students never exposed to the policy. If everyone is eligible and must take the exam, then all students will be treated at some point in time. But is this really the case? Maybe a subset of students in a particular region just missed the deadline to sit for the exam. Or, maybe the policy went into effect at different times in different schools/universities across the country. Only you can answer these questions. At a basic level, your method requires observation of a treatment group and a control group both before and after administration of the exam.

I want to use the students who have long been keeping zero spendings on private education as a control group?

This approach seems untenable in my estimation and ad hoc. To return to my question in the comments, students without any expenditures would still be impacted by the policy. What you are doing is constructing a counterfactual group of students without any private investments. Selecting a control group that is very low in your outcome measure is not something I would recommend.

Another way to see how this is problematic, given the methodology you're espousing, is to plot the inter-temporal evolution of "mean expenditures" of your exposed (i.e., some expenditure) and unexposed (i.e., zero expenditure) groups in the pre-treatment period. Are these trends parallel? Is the non-expenditure group a suitable counterfactual for the subset of students with actual investments? Your treatment effect would likely be biased if the exposed group is already trending upward/downward in the pre-treatment epoch. In my estimation, the 'zero-investment' group is not a suitable representation of what would have happened in the absence of treatment. And, if I understand you correctly, students without any private expenditures over time would still be impacted by the policy.

Do you know any study that uses this kind of control group?

Not in my experience. This is a pretty big forum so others might jump in and offer their insight. What I can comment on is a difference-in-differences analysis requires you to observe—before and after treatment—at least some group of students who were never subjected to the policy. My recommendation is to find a region of the country where there may have been early/late adopters of treatment. Or, try looking for more data in a neighboring country.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.