1
$\begingroup$

Within the context of using R package CausalImpact (Brodersen et. al, 2015), is it valid to use time Periods that are noncontinuous for the Pre-Intervention and Post Intervention time periods?

For example, would it be be valid to use Pre-Intervention Time periods: '2017-01-02' to '2018-03-22' and Post-Intervention time period: '2018-04-01' to '2018-04-30'?

For further context, as a form of backtesting I tested the time period of '2018-03-15' to '2018-03-31' for Casual Impact using all the same covariates and saw a high probability of Impact, but this impact was negative (False Positive of a negative impact). However, in theory, we should of not seen a chance of impact above our significance level as no treatment was applied (our actual treatment was applied for 2018-04-01 to 2018-04-30). So, in an effort to reduce bias the results I wanted to exclude this time period from the analysis.

$\endgroup$
  • $\begingroup$ Typically this FP result would invalidate the estimation strategy. If you are finding effects where theory predicts there should be none, what makes you think that this won't carry over to the intervention period, particularly if the periods are contiguous? Throwing this data away seems akin to discarding contradictory evidence. $\endgroup$ – Dimitriy V. Masterov May 16 '18 at 19:52
  • $\begingroup$ It is possible that if people change behavior in anticipation of treatment, that you would get results showing up before the treatment starts. One example is an early announcement of a sale that simply delays some purchases to the sale period. In this case, you can redefine treatment to be the announcement rather than the sale itself. $\endgroup$ – Dimitriy V. Masterov May 16 '18 at 19:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.