# Measuring the causal impact of a policy that is not binding

This may be a little tricky because it's difficult to explain but bear with me. Assume a new policy implemented in 2015 which is a new requirement for firms, let's say for instance that the requirement is to buy some insurance if they hold specific assets, something that was not necessary to do before this policy.

Firms can either simply start buying this insurance as required, or if they don't want to pay the insurance, they can stop using these assets.

In the data, we only observe firms who buy the insurance several years after the implementation of the reform, in 2019. I define two groups: those that have the insurance in 2019 ($$Insurance2019=1$$ and those that have not ($$Insurance2019=0$$).

Now I want to study whether we observe different dynamics in the balance sheet of firms (let's call this variable $$Y_{i}$$ for firm $$i$$) that have the insurance in 2019 vs. those that do not.

$$Y_{it}=\delta_t \sum_{t=2011}^{2019}Year_{t}+\beta_t \sum_{t=2011}^{2019}Year_{t} \cdot Insurance2019_i + \epsilon_{it}$$

The idea is to detect whether firms that have the insurance in 2019 changed the dynamics of their balance sheet ($$Y$$) from 2015 onwards compared to those that do not have the insurance. The idea is that we should not see any difference in $$Y$$ for the two groups between 2011 and 2015, but then we should observe something.

My question:

• Does this model suffer from endogeneity? In an optimal setup of policy evaluation, we define treatment and control groups based on whether a firm is the target of the policy or not. Here it's a little different since firms can choose to avoid the policy requirement.
• In other words: using this model, if I find that $$beta_{t}$$ is significant from 2015 onwards, can I say something like: "Having the insurance in 2019 has had a causal impact on the dynamics of $$Y$$ from 2015 onwards" ?

Let $$I$$ be the variable of whether a company takes the insurance policy or not, and let $$Y$$ be the effect you wish to measure. Let $$U_Y$$ be the exogenous variable giving rise to the error term. Then a causal diagram for you situation would be like this:

We know that $$I$$ causally affects $$Y$$, as does $$U_Y.$$ The dotted line between $$I$$ and $$U_Y$$ indicates an unknown relationship, with correlation as its symptom. You want to know if there is an endogeneity problem. The answer is that it depends. The relationship between $$I$$ and $$U_Y$$ could be this:

In this case, there is no endogeneity problem at all, because there is no backdoor path from $$I$$ to $$Y.$$ On the other hand, you could have this:

Now you have a backdoor path from $$I$$ to $$Y,$$ and you have an endogeneity problem.

Given that (I assume) you have not measured $$U_Y,$$ you are likely going to need to adjust for it somehow or other, just in case it is a problem. In that case, you have two options of which I am aware. The first is instrumental variables. The idea is this: introduce another measured variable $$Z$$ that has $$I$$ as its child, and does NOT have $$U_Y$$ or $$Y$$ as its child:

Then you find ratios of regression coefficients to find the right causal effect of $$I$$ on $$Y,$$ a procedure outlined on pages 85-88 of Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

The second approach would be to insert a measured variable $$Z$$ in-between $$I$$ and $$Y$$ thus:

Now you can use the front-door adjustment formula on page 68 of the same book: $$P(Y=y|\operatorname{do}(I=i))=\sum_z\sum_{i'}P\big(Y=y|Z=z,I=i'\big)\,P\big(I=i'\big)\,P(Z=z|Z=i).$$

• Thanks! I will check the book. I was surprised to read the following: "We know that I causally affects $Y$, as does $U_Y$", is it possible to say that residuals causally impact the outcome variable? Thanks again! – user6441253 Apr 23 '20 at 8:15
• I just mean that $U_Y$ is the exogenous variable giving rise to your error term $\epsilon_{it},$ that's all. It contributes to $Y,$ as it is in your model. – Adrian Keister Apr 23 '20 at 13:54
• Ok I understand, thanks. – user6441253 Apr 23 '20 at 15:13