2
$\begingroup$

Imagine the following situation to illustrate my question:

We want to run an RCT to test whether policy2, combined with policy1, is better than just policy1 alone.* On top of that we want to see whether the version A or version B of policy2 is more effective (stronger effect). Thus, is it ok if we consider the following three treatments?

Treatment1: Policy1+ Policy2a

Treatment2: Policy1+ Policy2b

Control: Policy1

Then, is it okay to pick Policy1 as a control, then estimate a simple OLS to get the treatment effect with two indicators for Treatment1 and Treatment2? Assume all groups are balanced.

*We assume that Policy1 is already somehow effective by relying on previous literature but don't demonstrate it via a comparison with a pure control group. For the sake of robustness, should we even so prove it in our sample?

$\endgroup$

1 Answer 1

1
$\begingroup$

Based on your description, you have correctly specified your groups.

You are effectively doing two separate experiments with a shared control group. If you are testing two separate hypotheses, then two separate tests would be more appropriate.

In addition, if you use one model for both comparisons, your test statistics will be slightly lower (p-values higher) than if you used two independent tests for each hypothesis. This is due to a higher degree of freedom when using a single regression model for both treatments.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.