7
$\begingroup$

There is a paragraph on interactions in The Book of Why (Pearl & Mackenzie, 2018), Chapter 9 (I cannot share the page number because I have the book in epub format), where the authors argue that:

However, Equation 9.4 does hold automatically in one situation, with no apparent need to invoke counterfactuals. That is the case of a linear causal model, of the sort that we saw in Chapter 8. As discussed there, linear models do not allow interactions, which can be both a virtue and a drawback. It is a virtue in the sense that it makes mediation analysis much easier, but it is a drawback if we want to describe a real-world causal process that does involve interactions. [Emphasis mine]

The equation 9.4 is

$$\text{Total Effect = Direct Effect + Indirect Effect}$$

They repeated a similar argument before in Chapter 8:

On the other hand, linear models cannot represent dose-response curves that are not straight lines. They cannot represent threshold effects, such as a drug that has increasing effects up to a certain dosage and then no further effect. They also cannot represent interactions between variables. For instance, a linear model cannot describe a situation in which one variable enhances or inhibits the effect of another variable. (For example, Education might enhance the effect of Experience by putting the individual in a faster-track job that gets bigger annual raises.)[Emphasis mine]

And in Chapter 7:

Keep in mind also that the regression-based adjustment* works only for linear models, which involve a major modeling assumption. With linear models, we lose the ability to model nonlinear interactions, such as when the effect of X on Y depends on the level of Z. The back-door adjustment, on the other hand, still works fine even when we have no idea what functions are behind the arrows in the diagrams. But in this so-called nonparametric case, we need to employ other extrapolation methods to deal with the curse of dimensionality. [Emphasis mine]

Why Pearl & Mackenzie argue that linear models do not allow interactions? Do I overlook an important detail and context-specific information?


*By regression-based adjustment, authors refer to (in the preceding paragraphs), what we sometimes call, "controlling for" other variables: "The analogue of a regression line is a regression plane, which has an equation that looks like $Y=aX+bZ+c$ ... The coefficient $a$ gives us the regression coefficient of $Y$ on $X$ already adjusted for $Z$. (It is called a partial regression coefficient and written $r_{YX.Z}$.)"

$\endgroup$
  • $\begingroup$ your quotes just highlight the question, can you provide information on eg what regression based adjustment is. $\endgroup$ – seanv507 Jun 17 at 8:47
  • $\begingroup$ By regression-based adjustment, authors refer to (in the preceding paragraphs), what we sometimes call, "controlling for" other variables: "The analogue of a regression line is a regression plane, which has an equation that looks like $Y = aX + bZ + c$... The coefficient $a$ gives us the regression coefficient of $Y$ on $X$ already adjusted for $Z$.(It is called a partial regression coefficient and written $r_{YX.Z}$.)" $\endgroup$ – T.E.G. Jun 17 at 9:02
  • $\begingroup$ so maybe add this to question... afaik this could still be done in theory for interaction terms, but is not typically done in practise $\endgroup$ – seanv507 Jun 17 at 9:08
5
$\begingroup$

You are conflating linear in parameters with linear in variables. Linearity here refers to the relationship between the variables.

Their point in the book is that, if the model is not linear in the variables, then neither the equation

$$\text{Total Effect} = \text{Direct Effect} + \text{Indirect Effect} $$

holds, nor the regression coefficient gives you the proper backdoor adjustment directly.

Regarding the last case, for instance, consider the conditional expectation $E[Y|x,z] = \beta x + \gamma z$, which is linear with respect to $X$ and $Z$.

If $Z$ satisfies the backdoor criterion for the causal effect of $X$ on $Y$, then

$$ \frac{\partial E[Y|do(x)]}{\partial x} = \frac{\partial E[E[Y|x, Z]]}{\partial x} = \beta $$

That is, the regression coefficient $\beta$ equals the average marginal causal effect. This is what is meant by "regression based adjustment works" in this case, you don't need any extra steps here---all the averaging required for backdoor adjustment is automatically done by regression.

Now consider the conditional expectation $E[Y|x,z] = \beta x + \gamma z + \delta (x \times z)$. Note this is not linear with respect to $x$ and $z$ (although it is linear in the parameters).

Note in this case if $Z$ satisfies the backdoor criterion for the causal effect of $X$ on $Y$, then

$$ \frac{\partial E[Y|do(x)]}{\partial x} = \frac{\partial E[E[Y|x, Z]]}{\partial x} = \beta + \delta E[z] $$

That is, the correct backdoor adjustment is not given by the regression coefficient on $X$ only.

More generally, Pearl is saying that if $Z$ satifies the backdoor criterion, you can use any non-parametric estimator you prefer to compute the post-intervention distribution $ E[Y|do(x)] = E[E[Y|x, Z]]$.

$\endgroup$
  • $\begingroup$ Thank you, @CarlosCinelli. I know your interest in Pearl’s work from this thread (stats.stackexchange.com/a/376925/109647) and I am glad you had time to write an answer here. It is more detailed compared to previous answer, but basically agrees with it. So by linear model, Pearl means linear in variables, but not in parameters. But, here is my issue: the term “linear” in linear model does not refer to being linear in variables. As far as I know, it never does… $\endgroup$ – T.E.G. Jun 18 at 14:58
  • $\begingroup$ As stated in this answer (stats.stackexchange.com/a/8706/109647), “linear refers to the relationship between the parameters that you are estimating and the outcome.” That’s one of the first things I learned in a regression course; you can model nonlinear relations (e.g., polynomial terms) in linear regression. We preserve the term nonlinear for those models which are not linear in parameters (e.g., $y= e^{\beta} + \varepsilon$)… $\endgroup$ – T.E.G. Jun 18 at 14:58
  • $\begingroup$ It seems to me that I am not conflating anything, only asking for clarification. Both answers boil down to this: Pearl means something else by “linear model.” But I have no reason to adopt the use of term Pearl prefers. And linear models (as in linear in parameters) do allow interactions. If what I overlook is only the different way “linear model“ used here, then I will accept this answer. $\endgroup$ – T.E.G. Jun 18 at 14:59
  • $\begingroup$ Hi @T.E.G. the answer you cite is talking about regression models. Here we are talking about causal (structural) models. The structural equation y = f(x, z) is linear if f(x,z) is a linear function of x and z. You may be able to estimate f(x,z) with OLS and variable transformations, but f(x,z) is still not linear. A structural model is said to be linear if all functions are linear. This is not just a difference in semantics--if the structural model is not linear then, as Pearl says: (1) backdoor adjustment differs from regression adjustment; and (2) the decomposition TE = DE + IE does not hold $\endgroup$ – Carlos Cinelli Jun 19 at 5:03
  • $\begingroup$ "A structural model is said to be linear if all functions are linear" in variables, right? So, a model like $Y= \beta_0 +\beta_1X_1 + \beta_2X_2 + \beta_3(X_1 \times X_2) + \varepsilon$ is not linear in causal modeling framework. $\endgroup$ – T.E.G. Jun 19 at 5:39
2
$\begingroup$

"Purely linear" models do not allow for that. If you want to model an interaction using a particular case of the General Linear Model (do not mistake this for a Generalized Linear Model), you have to introduce an artificial extra variable like the product of the two interacting ones.

This new model is still linear with regards to its parameters (this is what matters for getting the estimators), but it is no longer linear with regards to its variables (you can no longer talk about a linear relationship between regressors and target)

$\endgroup$
  • 1
    $\begingroup$ Thank you. What is a "purely linear" model? Also, the authors did not use such a term in the book. $\endgroup$ – T.E.G. Jun 17 at 7:49
  • 1
    $\begingroup$ They didn't. I just made it up. I refer to models that are linear with regards to both variables and parameters $\endgroup$ – David Jun 17 at 7:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.