2
$\begingroup$

I'm working on causal inference using Interrupted Time Series Design. I have multiple samples per day and am selecting my analysis bandwidth based on pre-treatment RMSE on leave-on-out cross validation. I have both a treatment and a control group, which I use to obtain the baseline trends. The data is already 0 centered, with 0 being the date in which treatment/placebo administration began.

The catch is that for both of my groups, I have an uneven number of samples each day, and the distribution of those sample are also markedly different, as per the plot below:

enter image description here

How should I handle building the ITS regression model? Is it proper to disregard the difference in sample frequencies and go ahead with all samples? Should I instead downsample all days to match the lowest day? Should I create a single sample per day by taking daily averages (or medians)?

My ITS model is (what I believe to be) the standard one, with a single dependent variable and has as independent variables time, exposed (a dummy for treatment/control), interrupted (a dummy for pre/post treatment), and all their interaction terms.

$\endgroup$

1 Answer 1

1
$\begingroup$

The first obvious thing to deal with here is the question --- Why do the control and treatment group look so different (and why does the treatment groups suddenly trend upward rapidly) before the treatment is even applied? This suggests either that you have misstated the start of the treatment, or you have a fundamental problem in your trial where the treatment and control groups are systematically different before the treatment is even applied. If I were you, I would put any considerations of modelling on the back-burner until you have given attention to this issue.

$\endgroup$
2
  • $\begingroup$ Ben, thanks for the reply. I tried to kept the description not overly long to spare people from details I though were not relevant to the issue of unevely distributed samples. But let me share a few additional details. This analysis is ex-post-facto, where we we looking at how the hate speech of users changes after they join a know hateful subreddit. We used mahalanobis matching to get control users as similar as possible given our feature set, but obviously the plot shows they are still less active. (1/2) $\endgroup$ Feb 4, 2022 at 3:07
  • $\begingroup$ Day 0 is the day the treatment users join the hateful subreddit, and for control users day 0 is the day their "matched treatment" joined the subreddit. It seems to me to be that the treatment users are much more active around their join date, such that more of them as posting on more days. Given this, do you have any further advice? Once again, thanks for the input. (2/2) $\endgroup$ Feb 4, 2022 at 3:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.