Moderated regression: Why do we calculate a *product* term between the predictors?

Moderated regression analyses are often used in social sciences to assess the interaction between two or more predictors/covariates.

Typically, with two predictor variables, the following model is applied:

$Y = β_0 + β_1*X + β_2*M + β_3*XM + e$

Notice that the test of moderation is operationalized by the product term $XM$ (the multiplication between independent variable $X$ and moderator variable $M$). My very fundamental question is: why do we actually calculate a product term between $X$ and $M$? Why not, for instance, the absolute difference $|M-X|$ or just the sum $X + M$?

Interestingly, Kenny alludes to this issue here http://davidakenny.net/cm/moderation.htm by saying: "As will be seen, the test of moderation is not always operationalized by the product term XM" but no further explanation is given. A formal illustration or proof would be enlightening, I guess/hope.

A "moderator" affects the regression coefficients of $Y$ against $X$: they might change as values of the moderator change. Thus, in full generality, the simple regression model of moderation is

$$\mathbb{E}(Y) = \alpha(M) + \beta(M)X$$

where $\alpha$ and $\beta$ are functions of the moderator $M$ rather than constants unaffected by values of $M$.

In the same spirit in which regression is founded on a linear approximation of the relationship between $X$ and $Y$, we may hope that both $\alpha$ and $\beta$ are--at least approximately--linear functions of $M$ throughout the range of values of $M$ in the data:

\eqalign{ \mathbb{E}(Y) &= \alpha_0 + \alpha_1 M + O(M^2) + (\beta_0 + \beta_1 M + O(M^2))X \\ &= \alpha_0 + \beta_0 X + \alpha_1 M + \beta_1 MX + O(M^2) + O(M^2)X. }

Dropping the nonlinear ("big-O") terms, in the hope they are too small to matter, gives the multiplicative (bilinear) interaction model

$$\mathbb{E}(Y) = \alpha_0 + \beta_0 X + \alpha_1 M + \beta_1 MX.\tag{1}$$

This derivation suggests an interesting interpretation of the coefficients: $\alpha_1$ is the rate at which $M$ changes the intercept while $\beta_1$ is the rate at which $M$ changes the slope. ($\alpha_0$ and $\beta_0$ are the slope and intercept when $M$ is (formally) set to zero.) $\beta_1$ is the coefficient of the "product term" $MX$. It answers the question in this way:

We model the moderation with a product term $MX$ when we expect the moderator $M$ will (approximately, on average) have a linear relationship with the slope of $Y$ vs $X$.

Of interest is that this derivation points the way towards a natural extension of the model, which might suggest ways to check goodness of fit. If you are not concerned with nonlinearity in $X$--you either know or assume that model $(1)$ is accurate--then you would want to extend the model to accommodate the terms that were dropped:

$$\mathbb{E}(Y) = \alpha_0 + \beta_0 X + \alpha_1 M + \beta_1 MX + \alpha_2M^2 + \beta_2 M^2X.$$

Testing the hypothesis $\alpha_2=\beta_2=0$ evaluates the goodness of fit. Estimating $\alpha_2$ and $\beta_2$ could indicate in what way model $(1)$ might need to be extended: to incorporate nonlinearity in $M$ (when $\alpha_2 \ne 0$) or a more complicated moderating relationship (when $\beta_2 \ne 0$) or possibly both. (Note that this test would not be suggested by a power series expansion of a generic function $f(X,M)$.)

Finally, if you were to discover that the interaction coefficient $\beta_1$ were not significantly different from zero, but that the fit is nonlinear (as evidenced by a significant value of $\beta_2$), then you would conclude (a) there is moderation but (b) it is not modeled by an $MX$ term, but instead by some higher-order terms beginning with $M^2X$. This might be the kind of phenomenon to which Kenny was referring.

If you use the sum of predictors to model their interaction, your equation would be:

$$\begin{eqnarray} Y &=& \beta_0 + \beta_1X + \beta_2M + \beta_3(X + M) + e\\ &=& \beta_0 + \beta_1X + \beta_2M + \beta_3X + \beta_3M + e\\ &=& \beta_0 + (\beta_1 + \beta_3)X + (\beta_2 + \beta_3)M + e \\ &=& \beta_0 + \beta_1'X + \beta_2'M + e \end{eqnarray}$$

where $\beta_1'=\beta_1+\beta_3$ and $\beta_2'=\beta_2+\beta_3$. Therefore, your model would have no interaction at all. Clearly, this is not the case with product.

Recall the definition of the absolute value:

$$|X-M| = \begin{cases} X-M, & X \geq M\\ M-X, & X < M \end{cases}$$

Although you can reduce the model $\beta_0 + \beta_1X + \beta_2M + \beta_3|X-M| + e$ to the one with only $X$ and $M$ terms, using the def. of $|X-M|$, the absolute value is a "specialized form of moderation that is unlikely to be realistic in many situations", as pointed out in the comment below.

• Actually, including an $|X-M|$ term is demonstrably a form of moderation: the value of $M$ changes $\beta_2$. It is, however, a limited, specialized form of moderation that is unlikely to be realistic in many situations. It is not correct to say that such a model has "only main effects." – whuber Jul 21 '16 at 16:30
• Yes, you are right, $|X-M|$ is a form of moderation, I got carried away by transformation and will edit the answer accordingly. Thanks for pointing this out. – Milos Jul 21 '16 at 16:48
• @Milos: Your example about the sum of predictors was an eye-opener, a somewhat embarrassing one, I must say because I should have already realized the mathematical implications ;) whuber: As far as I understand it, the absolute value is only useful when both predictor variables are measured in same units (e.g. two psychometric tests, using the same metric, such as z-scores or T-scores). The absolute difference between X and M is a useful metric, although not the only possible one (i.e. the prodcut term could also be used). – denominator Jul 22 '16 at 21:16

You won't find a formal proof for using multiplicative moderator. You can support this approach by other means. For instance, look at the Taylor-MacLaurin expansion of a function $f(X,M)$:

$$f(X,M)=f(0,0)+\frac{\partial f(0,0)}{\partial T} T+\frac{\partial f(0,0)}{\partial M} M+\frac{\partial^2 f(0,0)}{\partial T\partial M} TM +\frac{\partial^2 f(0,0)}{2\partial T^2} T^2 +\frac{\partial^2 f(0,0)}{2\partial M^2} M^2\dots$$

If you plug a function of this form $f(X,M)=\beta_0+\beta_XX+\beta_MM+\beta_{XM}XM$ into the Taylor equation, you get this:

$$f(X,M)=\beta_0+\beta_XX +\beta_MM +\beta_{XM}XM$$

So, the rationale here is that this particular multiplicative form of the moderation is basically a second order Taylor approximation of a generic moderation relationship $f(X,M)$

UPDATE: if you include quadratic terms, as @whuber suggested then this will happen: $$g(X,M)=b_0+b_XX +b_MM +b_{XM}XM+b_{X2}X^2 +b_{M2}M^2$$ plug this into Taylor: $$g(X,M)=b_0+b_XX +b_MM +b_{XM}XM +b_{X2}X^2 +b_{M2}M^2$$

This shows that our new model $g(X,M)$ with quadratic terms corresponds to a full second order Taylor approximation, unlike the original moderation model $f(X,M)$.

• Since the basis of your argument is the Taylor expansion, why did you not also include the other two quadratic terms $X^2$ and $M^2$? True, they are not forms of moderation, but their inclusion in the model usually will affect $\beta_{XM}$. – whuber Jul 21 '16 at 16:32
• @whuber, I decided to keep the post short - that's the main reason. Otherwise, I started writing about my preference to include second order terms whenever you have a cross term, then cut it out. – Aksakal Jul 21 '16 at 17:06