Consider the following causal model:

enter image description here

For each of the parameters in the model, write a regression equation in which one of the coefficients is equal to that parameter. Identify the parameters for which more than one such equation exists.

My attempt:

For the first part of the problem, we can simply regress a variable on its parents. For example, if we regress $Z_3$ on $Z_1$ and $Z_2$ with the equation: $Z_3=R_{Z_1}Z_1+R_{Z_2}Z_2$ Then, $R_{Z_1}=a_3$ and $R_{Z_2}=b_3$.

For the second part of the problem, I would like to find whether there are other regression equations which allow us to find the same parameters.

The book defines a procedure called "The Regression Rule for Identification" to answer these kinds of questions. The procedure consists of the following: given a graphical model in which $X$ has a direct effect $\alpha$ on $Y$, we consider the graph $G_{\alpha}$ with the edge which goes from $X$ to $Y$ removed. Then we must find a set $Z$ which d-separates $X$ and $Y$ in $G_{\alpha}$, and the coefficient of $X$ in the regression of $Y$ on $X$ and $Z$ is $\alpha$.

I know that regressing a variable on its parents is a particular example of this rule. But for example, suppose we want to find another equation to determine $b_3$. Let us consider the graph $G_{b_3}$, in this graph $Z_2$ and $Z_3$ are d-separated without needing to condition on any other variable, as all the paths from $Z_2$ and $Z_3$ must go through a collider at $Y$. Therefore, $b_3$ can be identified by the regression $Z_3=R_{Z_2}·Z_2$.

Edit: After simulating this model, it seems that this application of the rule is indeed correct.

Now, my question is why adding $Z_1$ as a regressor does not change the regression coefficient? I was under the impression that the regression coefficients remain unaltered when adding new regressors only when the dependent variable is independent on the new regressor given the old regressors.


2 Answers 2


Since this is self-study question, let me give you such a hint:

What about trying to simulate such structure in statistical software? We can assume values and functional forms, and then check if regression gives right values.

Lets try to create such structure, in this example only for variables Z3, X, W3, and Y (we cut out the rest): An R code which creates such simulated data:

# Number of observations:
n = 10000

# Create Z3:
Z3 = rnorm(n)

# Create X:
t2 = 0.4
X = t2 * Z3 + rnorm(n)

# Create W3:
c3 = 0.5
W3 = c3 * X + rnorm(n)

# Create Y:
b = 0.7
a = 0.8 
Y = b * Z3 + a * W3 + rnorm(n)

Now lets try the regression: $Y_i = \beta_0 + \beta_1 W3_i + \varepsilon_i$

# Regressions:
m = lm(Y ~ W3)

In my version, the parameter $\beta_1$ was: 0.92191. Such value seems to be too high for a parameter $a$. How about regression $Y_i = \beta_0 + \beta_1 W3_i + \beta_2 X_i + \varepsilon_i$ then?

PS: If you are not sure about parameter values, you can always make them much different or increase number of observations. It is identification, so we can have infinite data!

  • 1
    $\begingroup$ I see, in your second regression the result is much better. Moreover, conditioning on $X$ makes $Y$ and $W_3$ d-separated in the graph $G_a$, so it follows the Regression Rule for Identification. $\endgroup$
    – NFC
    Oct 27, 2020 at 10:55
  • 1
    $\begingroup$ Now I have tried this using all the variables, and it seems that it is indeed possible to estimate $b_3$ by regressing $Z_3$ on just $Z_2$. I just have a hard time understanding why adding $Z_1$ to the regression does not change the regression coefficient for $Z_2$, considering that $Z_1$ and $Z_3$ are not independent given $Z_2$. $\endgroup$
    – NFC
    Oct 27, 2020 at 11:03
  • $\begingroup$ Great! Such simulations seem to give the 'answers', similar to answers to exercises for typical mathematical courses. Of course theory comes first, but in my opinion it is the best way to understand Causal Inference as I can think of. If my 'hint' is hintful enough, it would be great if you updated your question with results of your inquiries! $\endgroup$
    – cure
    Oct 27, 2020 at 12:55
  • $\begingroup$ Another (I guess mindblowing) hint: What would happen if in search for the parameter $b_3$ you tried regression: $Z3_i = \beta_0 + \beta_1 Z2_i + \beta_2 Y_i + \varepsilon_i$? The phrase conditioning on collider could be helpful. You can search scholar for the very friendly paper by Elwert and Winship "Endogenous selection bias: The problem of conditioning on a collider variable" to help you to understand causal graphs and regressions. $\endgroup$
    – cure
    Oct 27, 2020 at 13:01
  • 1
    $\begingroup$ When you condition on $Y$ you are unlocking some paths from $Z_2$ to $Z_3$ (such as $Z_2\rightarrow W_2 \rightarrow Y \leftarrow Z_3$). Therefore, since $Z_3$ and $Z_2$ are not independent given $Y$, the slope $\beta_1$ no longer represents the direct effect of $Z_2$ on $Z_3$, since it would include effects due to the other paths between $Z_3$ and $Z_2$ which are now open. That is what I think at least, I will look into the paper you mention. $\endgroup$
    – NFC
    Oct 27, 2020 at 19:31

For the question asked by NFC, I think it is because $Z_1$ and $Z_2$ are independent.

  • $\begingroup$ This would have been more appropriate as a comment. $\endgroup$
    – R Carnell
    Aug 25, 2022 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.