How do I test whether an intervention is significant when I don't have the counterfactual sample size?

Question

Let's say I want to figure out whether a specific Public Service Announcement (PSA; you can think of it as an advertisement) to ask people to do an action is effective. That is, the independent variable (IV) is PSA (or not) and the dependent variable (DV) is the proportion of people who act. Also assume that 1) the PSA can only be applied at the state level, 2) not all people in treated states would see the PSA, but I can find out how many have viewed the PSA, and 3) how many acted among the viewed. Because states are not true experimental intervention, I also want to test this using difference-in-difference. What I think I could do is something like

proportion ~ state + post + treatment*post

1. But while I can determine the number of people who actually saw the PSA (in the treated states), I don't know the number of the people who would have seen the PSA ads in the control or in the pre-period.
   • Can I run another regression proportion ~ state + post to create a counterfactual for post/control?
2. How do I calculate power (e.g. as a function of percentage of people who see the PSA, assuming it's the same across all treated states)?

Answer 1

It sounds to me that you have an IV setup. PSA is your instrument, i.e., treatment assignment, and the number of people who saw the PSA (exposure) is your treatment take-up. We might worry that exposure is endogenous but if PSA is randomized across states, you can use it as an instrument. In states without a PSA, exposure is zero.