3
$\begingroup$

Imagine a set of variables, X1, X2, and Y, all continuous variables. There is a simple case where X1 and X2 affect Y such that:

Y = alpha + β1 X1 + β2 X2 + error

Using R syntax, a model to analyze such data would be:

lm(Y ~ X1 + X2)

Consider a further case where the causal process is such that the effect of X1 on Y changes when X2 changes, but the reverse is not true (the effect of X2 on Y does not depend on X1).

Y = alpha + β1 X1 + β2 X2 + β3 X1 x X2 + error

I would think of a model like this (R notation) to analyze these data:

lm(Y ~ X1 + X2 + X1:X2)

Assume that β1, β2, β3 are all positive.

Is there a way to disentangle whether β1 increases with increasing X2 or whether β2 increases with increasing X1? In other words, how can we tell whether it is the effect of X1 that increases with X2 or the effect of X2 that increases with X1?


This part has been added after reading @PaulG 's answer.

Thank you so much for your response! Very interesting and practical.

What I meant to ask was based on a logic in which one predictor (X1 or X2) is assumed to be a moderator and the other to be a focal variable. As a result, if:

Y = β0 + β1 X1 + β2 X2 + β3 X1 x X2 (eq.1),

this equation can be restated in two ways.

First, assuming that X1 is the focal predictor and X2 is the moderator term, we can regroup terms by factoring out X1:

Y = (β0 + β2 X2) + (β1 + β3 X2) x X1 (eq.2),

where (β0 + β2 X2) represents the intercept and (β1 + β3 X2) describes the relationship between X1 and Y if X2 is held constant. This relationship changes as the value of X2 changes. The above part is explained in a similar way in a youtube video about the interpretation of the interaction of continuous variables in regressions. I am afraid this is imprecise mathematically, but conceptually I see it as a changing (according to X2) effect of X1 on Y. Am I mistaken?

If we assume that X2 is the focal predictor and X1 is the moderator, we can regroup eq.1 as follows:

Y = (β0 + β1 X1) + (β2 + β3 X1) x X2 (eq.3),

where (β0 + β1 X1) represents the intercept and (β2 + β3 X1) represents the effect of X2 on Y if X1 is held constant.

I can envision scenarios in which X1's effect on Y is dependent on X2, but X2's effect on Y is independent of X1. In this case, I would try to model the data using eq.1, which I know can be "restated" as eq.2 (which is appropriate for such a situation!). However, eq.1 can also be "restated" as eq. 3, which does NOT fit the generating process. Any solution to this?

$\endgroup$
7
  • $\begingroup$ I would suggest (i) use mathematical equation (e.g., multiplication) rather than use R notation (e.g., X1:X2) and (ii) use $\beta$ instead of $beta$. $\endgroup$
    – TrungDung
    Jan 19, 2022 at 12:17
  • $\begingroup$ OK, thanks for the suggestion. $\endgroup$
    – simone
    Jan 19, 2022 at 13:09
  • $\begingroup$ There's a typo at the end, should be "or the effect of X2 that increases with X1?", not "[...] with X2?". :) $\endgroup$
    – PaulG
    Jan 19, 2022 at 14:11
  • $\begingroup$ correct! thanks! $\endgroup$
    – simone
    Jan 19, 2022 at 14:45
  • 1
    $\begingroup$ In response to your edit (below the bar): What you did is simple algebraic reforming of the same equation. Of course $X_1$ changes the effect of $X_2$ - but not through $\beta_1$ nor $\beta_2$, rather only through $\beta_3$. You can see that e.g. in eq2 since $X_1$ simply multiplies by $\beta_1$, $X_2$ has nothing to do with it. Analogously for eq3. Your 3 equations are equivalent, hence "restating" them does not make any difference for the regression - they all fit the DGP. The differences are purely visual. $\endgroup$
    – PaulG
    Jan 19, 2022 at 17:14

2 Answers 2

4
$\begingroup$

I believe your questions can be split into 2 parts:

First, what do interaction effects tell us? In your last equation the interaction term $\beta_3 X_1 X_2$ captures how variables $X_1$ and $X_2$ interact with each other. The main effects of $X_1$ and $X_2$ are captured by $\beta_1$ and $\beta_2$, where e.g. $\beta_1$ (on population level) tells us how $y$ is affected by $X_1$ regardless of what $X_2$ is. Same for $\beta_2$. $\beta_3$ captures the interaction effect between $X_1$ and $X_2$, i.e. whether they amplify or diminish each other. It does not tell us which causes which, but only that if $\beta_3>0$ then an increase of $X_1$ or of $X_2$ will lead to a greater increase of $y$ the greater the other variable is, and vice-versa for $\beta_3<0$. Note that the main effects captured by $\beta_1$ and $\beta_2$ are not affected by the interaction, since by including the interaction term, the relations between the two variables is captured by this interaction term (provided the model is correctly specified). This is why it is important to include interaction terms, otherwise the effects would 'spill over' to $\beta_1$, $\beta_2$ and make them biased/inconsistent.

In short, $\beta_1$ does not increase with $X_2$ and $\beta_2$ does not increase with $X_1$ - the interaction between them is completely captured by $\beta_3$. In fact, that is the whole point of a regression: to decompose the dynamics of $y$ into deterministic parts, with each depending only on the variables next to them.

Second, how do I know the causality between two variables $X_1$ and $X_2$? In other words, which one is the cause and which is the effect? This discussion can become lengthy, for which I defer to this post as a start. But the short answer is you can't know causality through a mathematical/statistical test, only by logic and common sense. There are problems of endogeneity which means that dependent $y$ and independent variables $X$ influence each other mutually. There are test for exogeneity, which is a property that infers correct model specification. These are good keywords for further research. But what $y$ is, is really at the discretion of the researcher and the questions they ask. In your example, if you want to inspect the relation between $X1$ and $X2$, then consider a regression between only these two, regardless of $y$ - however it might not help you much.

For example, let's assume $y$ represents crop yields for a field, which is only dependent on the soil quality $X$, with little variation. Whether you regress $y$ on $X$ or $X$ on $y$ I would expect to get equally good predictions. However, you would never really ask "is the crop yield causing the good soil?" because it's obvious here. Or even better, replace soil quality with sunshine in the example above, to make the causal logic absolutely clear.

$\endgroup$
2
  • $\begingroup$ Thank you once more for your time; I truly appreciate it. I'm still not sure if a model fitted to the data can reveal a case in which the effect of X1 on Y is modulated by X2, but the effect of X2 on Y is not modulated by X1. What kind of model should it be? In that case, what parameters' estimates should I expect? $\endgroup$
    – simone
    Jan 19, 2022 at 19:04
  • $\begingroup$ @simone: your question is equivalent to "does $X_1$ cause $X_2$?". As stated, this cannot be discovered once the DGP is defined. To illustrate, you're asking if this constellation is possible to identify: 1) $X_1 \Rightarrow y$, 2) $X_2 \Rightarrow y$, 3) $X_2 \Rightarrow X_1 \Rightarrow y$, 4) $X_1 \nRightarrow X_2$. By 3) and 4): $X_2 \Rightarrow X_1$ but $X_1 \nRightarrow X_2$, so you are asking if it is possible to identify if $X_2$ is a function of $X_1$: $X_2=f(X_1)$. This is not possible (unless you know the DGP) since it is equally valid that $f^{-1}(X_2)=g(X_2)=X_1$. $\endgroup$
    – PaulG
    Jan 19, 2022 at 20:49
2
$\begingroup$

PaulG has it correct (+1). There is nothing magical about an interaction term. It's just the product of the two predictors and its coefficient must be interpreted symmetrically between them. As your revised question shows, moderation by either of your two predictors of the effect of the other on Y leads to fundamentally the same regression model.

Causal reasoning is tricky. Hernán and Robins have a superb book introducing it, "What If." They find it best to start with analyses that don't involve regression and only much later discuss roles of regression models. In particular, some regression models that might help with prediction can be misleading for causal inference. As you seem to be early in thinking about causal models, their book might be a good place to start.

In your situation, perhaps if there were a mediator M of the effect of X1 on Y (X1 -> M -> Y), you hypothesized that X2 moderates M, and you could control (or control for) M separately from X2, then you might dissect apart "direct" effects of X2 on Y from those happening through its moderation of the X1 -> M -> Y axis. But that's a different model and study design from what you show.

$\endgroup$
2
  • $\begingroup$ Indeed I am early in causal models and I appreciate your reference to Hernan & Robin's book. My hypothesis is that Y depends on X1 and that the effect of X1 on Y is moderated by X2. X2 has not a causal effect on Y. If the DGP followed my hypothesis and I used a model like {Y ~ X1 + X2 + X1xX2}, I should get positive estimates for β1, β2, and β3. However, this is compatible also with the hypothesis that Y depends on X2 and that the effect of X2 on Y is modulated by X1. Is there any other solution to this problem besides using causal models or experimental approaches (impossible in my case)? $\endgroup$
    – simone
    Jan 19, 2022 at 21:50
  • $\begingroup$ @simone not that I can think of. Also, be careful when interpreting coefficients for individual predictors involved in interactions. Each of those individual coefficients then represents a difference from the Intercept with respect to the corresponding predictor when its interacting predictor is at 0 (continuous) or its reference level (factor). If you change the reference level/centering of an interacting predictor, the value and apparent "significance" of another predictor can change. See this page for a worked-through example. $\endgroup$
    – EdM
    Jan 19, 2022 at 22:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.