Promotion Effectiveness lacking a control group I want to measure the effectiveness of a promotion but I'm lacking a true control group. The data I have consists of a user base that can opt into signing up for a promotion (every user has access to the promotion). Currently, I am using users who opted into the promotion as my test group and those that did not as my control. My thought was using Google's Causal Impact to determine the counterfactual, but I am worried that I'm not capturing the underlying relationship between the two groups of users. Just wanted to see if my thought process is correct, or should I be using a different technique.
 A: You didn't mention what you are actually measuring as the result of the promotion, but let's say it is something like sales.
Your data is currently not really useful to measure the causal effect of a promotion on sales. It is very likely that you have hidden confounders in your scenario: E.g., it could be that only very engaged users opted into the promotion and those would have generated higher sales anyway. The confounder here would be the property of the user to be engaged or not. Then you would obtain a correlation between promotion and increased sales without an actual effect of promotion.
One standard way of circumventing those problems is by doing proper randomization, i.e. don't let the users decide about the promotion but assign the promotion yourself, and do so really randomly.
Google's CausalImpact requires you to train a model for sales before the intervention, i.e. all participants are without the promotion, then to intervene on all participants, i.e. give all of them the promotion, and then compare the predictions of sales given by the above model (i.e. how the sales would have continued without promotion) with the actual sales after the promotion. Again, this doesn't seem to fit your current data.
