0
$\begingroup$

Why do we use multiplication instead of another function to include interactions in linear regression?

$\endgroup$
5
  • 1
    $\begingroup$ Because an interaction between two variables is, literally, the product of those two variables. $\endgroup$ Sep 5, 2020 at 18:07
  • $\begingroup$ And why is the product of two variables considered a good representation of their interaction in the real world? $\endgroup$
    – numberfive
    Sep 5, 2020 at 18:10
  • $\begingroup$ It would help if you gave an alternative for us to discuss. $\endgroup$
    – Dave
    Sep 5, 2020 at 18:21
  • $\begingroup$ Okay. It's just that I've been introduced to the topic and they mentioned these interaction terms that are multiplicative, but I don't get the intuition behind that. I don't know. Maybe there are additive interactions? Power interactions? $\endgroup$
    – numberfive
    Sep 5, 2020 at 18:41
  • $\begingroup$ See stats.stackexchange.com/questions/41071, stats.stackexchange.com/questions/469117, and stats.stackexchange.com/questions/303716 for related discussions. $\endgroup$
    – whuber
    Sep 5, 2020 at 21:07

3 Answers 3

2
$\begingroup$

Let's say that at first your model is $y=\beta_0+\beta_1 x_1+\gamma x_2+e$, but $\gamma$, the partial effect of $x_2$ on $y$, might depend on $x_1$: $$\gamma = \beta_2+\beta_3 x_1$$ So you substitute $\gamma$ into your model: $$y=\beta_0+\beta_1 x_1 + (\beta_2+\beta_3 x_1)x_2+e=\beta_0 + \beta_1 x_1 + \beta_2 x_2+\beta_3 x_1x_2+e$$ This is where multiplication comes from.

EDIT

See whuber's answer to In multiple regression, why are interactions modelled as products, and not something else, of the predictors? for a more complete and more detailed explanation.

$\endgroup$
1
  • $\begingroup$ Excellent, thank you. That's super clear. $\endgroup$
    – numberfive
    Sep 5, 2020 at 19:45
1
$\begingroup$

Interaction is in reality more complex than the simple product term would suggest.

The true conditional mean function $f(x_1, x_2) = E(Y | X_1 = x_1, X_2 = x_2)$ is most certainly not a simply planar function except in the most primitive circumstances. In other words, you can generally assume that $f(x_1,x_2) \neq \beta_0 + \beta_1 x_1 + \beta_2 x_2$ for all $(x_1,x_2)$ of interest.

Thus, more complex functions are needed. A quick nod in that direction is the standard multiplicative interaction model where $f(x_1,x_2)$ is assumed equal to $\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1x_2$. Although this is a better approximation to the true $f(x_1, x_2)$, it is also most certainly not equal to $f(x_1,x_2)$, essentially for the same reason as for the planar (no-interaction) function.

There are classes of functions known as "universal approximators" that become closer to the true $f(x_1, x_2)$ as the complexity of the function increases. One of these classes motivates the simple product interaction; this is the polynomial class. Within this class, however, you also have all other product terms like $x_1x_2^2$, $x_1^3x_2^2$, etc. So from the standpoint of polynomial universal approximators, the simple product interaction model is the simplest level of complexity that can be introduced to incorporate interaction. But one can assume that all other polynomial product terms are also needed (at least in theory; estimation is a separate issue).

Other universal approximator classes exist and (necessarily) allow interactions, although these have quite different forms than the simple product term. Examples include neural network and Fourier function classes.

$\endgroup$
0
$\begingroup$

One reason is ease of interpretation. Say $$ y = x_1\beta_1 + x_2\beta_2 + x_1 x_2 \beta_{12} + \epsilon $$ We can rewrite as $$ y = x_1\beta_1 + x_2(\beta_2 + x_1 \beta_{12}) + \epsilon $$ For fixed $x_1$, we have a linear dependence on $x_2$, with intercept $x_1\beta_1$ and slope $\beta_2 + x_1 \beta_{12}$.

$\endgroup$
1
  • $\begingroup$ Thank you for your answer, Wicher. $\endgroup$
    – numberfive
    Sep 5, 2020 at 19:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.