# The Regression Rule for Identification: problem 3.8.1(c) in Causal Inference in Statistics: A Primer

Consider the following causal model:

For each of the parameters in the model, write a regression equation in which one of the coefficients is equal to that parameter. Identify the parameters for which more than one such equation exists.

My attempt:

For the first part of the problem, we can simply regress a variable on its parents. For example, if we regress $$Z_3$$ on $$Z_1$$ and $$Z_2$$ with the equation: $$Z_3=R_{Z_1}Z_1+R_{Z_2}Z_2$$ Then, $$R_{Z_1}=a_3$$ and $$R_{Z_2}=b_3$$.

For the second part of the problem, I would like to find whether there are other regression equations which allow us to find the same parameters.

The book defines a procedure called "The Regression Rule for Identification" to answer these kinds of questions. The procedure consists of the following: given a graphical model in which $$X$$ has a direct effect $$\alpha$$ on $$Y$$, we consider the graph $$G_{\alpha}$$ with the edge which goes from $$X$$ to $$Y$$ removed. Then we must find a set $$Z$$ which d-separates $$X$$ and $$Y$$ in $$G_{\alpha}$$, and the coefficient of $$X$$ in the regression of $$Y$$ on $$X$$ and $$Z$$ is $$\alpha$$.

I know that regressing a variable on its parents is a particular example of this rule. But for example, suppose we want to find another equation to determine $$b_3$$. Let us consider the graph $$G_{b_3}$$, in this graph $$Z_2$$ and $$Z_3$$ are d-separated without needing to condition on any other variable, as all the paths from $$Z_2$$ and $$Z_3$$ must go through a collider at $$Y$$. Therefore, $$b_3$$ can be identified by the regression $$Z_3=R_{Z_2}·Z_2$$.

Edit: After simulating this model, it seems that this application of the rule is indeed correct.

Now, my question is why adding $$Z_1$$ as a regressor does not change the regression coefficient? I was under the impression that the regression coefficients remain unaltered when adding new regressors only when the dependent variable is independent on the new regressor given the old regressors.

## 2 Answers

Since this is self-study question, let me give you such a hint:

What about trying to simulate such structure in statistical software? We can assume values and functional forms, and then check if regression gives right values.

Lets try to create such structure, in this example only for variables Z3, X, W3, and Y (we cut out the rest): An R code which creates such simulated data:

# Number of observations:
n = 10000

# Create Z3:
Z3 = rnorm(n)

# Create X:
t2 = 0.4
X = t2 * Z3 + rnorm(n)

# Create W3:
c3 = 0.5
W3 = c3 * X + rnorm(n)

# Create Y:
b = 0.7
a = 0.8
Y = b * Z3 + a * W3 + rnorm(n)

Now lets try the regression: $$Y_i = \beta_0 + \beta_1 W3_i + \varepsilon_i$$

# Regressions:
m = lm(Y ~ W3)
summary(m)

In my version, the parameter $$\beta_1$$ was: 0.92191. Such value seems to be too high for a parameter $$a$$. How about regression $$Y_i = \beta_0 + \beta_1 W3_i + \beta_2 X_i + \varepsilon_i$$ then?

PS: If you are not sure about parameter values, you can always make them much different or increase number of observations. It is identification, so we can have infinite data!

• I see, in your second regression the result is much better. Moreover, conditioning on $X$ makes $Y$ and $W_3$ d-separated in the graph $G_a$, so it follows the Regression Rule for Identification.
– NFC
Oct 27, 2020 at 10:55
• Now I have tried this using all the variables, and it seems that it is indeed possible to estimate $b_3$ by regressing $Z_3$ on just $Z_2$. I just have a hard time understanding why adding $Z_1$ to the regression does not change the regression coefficient for $Z_2$, considering that $Z_1$ and $Z_3$ are not independent given $Z_2$.
– NFC
Oct 27, 2020 at 11:03
• Great! Such simulations seem to give the 'answers', similar to answers to exercises for typical mathematical courses. Of course theory comes first, but in my opinion it is the best way to understand Causal Inference as I can think of. If my 'hint' is hintful enough, it would be great if you updated your question with results of your inquiries!
– cure
Oct 27, 2020 at 12:55
• Another (I guess mindblowing) hint: What would happen if in search for the parameter $b_3$ you tried regression: $Z3_i = \beta_0 + \beta_1 Z2_i + \beta_2 Y_i + \varepsilon_i$? The phrase conditioning on collider could be helpful. You can search scholar for the very friendly paper by Elwert and Winship "Endogenous selection bias: The problem of conditioning on a collider variable" to help you to understand causal graphs and regressions.
– cure
Oct 27, 2020 at 13:01
• When you condition on $Y$ you are unlocking some paths from $Z_2$ to $Z_3$ (such as $Z_2\rightarrow W_2 \rightarrow Y \leftarrow Z_3$). Therefore, since $Z_3$ and $Z_2$ are not independent given $Y$, the slope $\beta_1$ no longer represents the direct effect of $Z_2$ on $Z_3$, since it would include effects due to the other paths between $Z_3$ and $Z_2$ which are now open. That is what I think at least, I will look into the paper you mention.
– NFC
Oct 27, 2020 at 19:31

For the question asked by NFC, I think it is because $$Z_1$$ and $$Z_2$$ are independent.

• This would have been more appropriate as a comment. Aug 25, 2022 at 16:18