Why is the interaction term defined by multiplying the IVs? I am trying to get a deeper mathematical understanding of regression, beyond just applying it and interpreting the output. Can somebody provide me with the mathematical rationale for why interaction terms in a regression equation are defined by the product of the involved independent variables? I assume it has to do with calculating the model sum of squares, and the portion of variance in the IV which cannot be independently attributed to the IV alone (as a main effect), but I do not understand the underlying mathemathics.
 A: It's easiest to think of this in terms of categorical predictors. Let $Y$ = sweetness of a cup of a coffee measured in some way. Let $X_1$ = $1$ if you stirred the coffee, and $0$ if you did not stir the coffee. Let $X_2$ = $1$ if you added sugar to the coffee, and $0$ if you did not add sugar to the coffee.
Now suppose you posit the model :
$$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \epsilon$$
What does $\beta_1$ represent? It represents the effect that stirring the coffee has on its sweetness. But that seems strange by itself. Of course, the effect of stirring on coffee depends on whether or not sugar was added. So the betas, as it stands, don't really capture the relationship as it exists in reality. Let's modify our model as such:
$$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3(X_1 \cdot X_2) + \epsilon$$
So we've added an interaction term. What is the effect of this modification? Well, $\beta_3(X_1 \cdot X_2)$ will be $0$ if either $X_1$ or $X_2$ are $0$. It only affects calculation of $Y$ if their product is $1$, i.e. $X_1 = X_2 = 1$, i.e. you both added sugar and stirred the coffee. So we've allowed for an extra effect on sweetness when both predictors take value $1$ ("the whole is more than the sum of its parts"), and we've done so by taking advantage of properties of the product of two binary variables.
This is why it's defined this way. Interactions between continuous variables are also defined by a product. Try to think of an example with two continuous predictors, and what happens to their product given high/low values of each.
A: Here is an approach. Consider this model: $$Y = \beta_0+\beta_1 X_1 + \beta_2 X_2 + \varepsilon.$$ Now, the marginal effect of $X_1$ on $Y$ ($Y$'s derivative w.r.t $X_1$) is simply $\beta_1$. I can replace $\beta_1$ with anything, that will be $X_1$'s effect (assuming it doesn't contain $X_1$): $$Y = \beta_0+ \left(<\text{what we write here will be $X_1$'s effect}>\right) X_1 + \beta_2 X_2 + \varepsilon.$$
In the first case, we wrote $\beta_1$, which is a constant, so the effect of $X_1$ is independent of $X_2$. That's why there is no interaction. We want to introduce an interaction, i.e., that $X_1$'s effect depends on $X_2$? Let's write something there which depends on $X_2$! For simplicity, be linear here as well, i.e., assume that the dependence itself is also linear: $$Y = \beta_0+\left(\beta_1 + \gamma_{12}X_2\right) X_1 + \beta_2 X_2 + \varepsilon.$$ That is, the effect of $X_1$ will be $\beta_1 + \gamma_{12}X_2$, so we have the interaction as we aimed for: the effect of $X_1$ depends on the value of $X_2$. (In this simplest case, linearly.)
Now we expand it: $$\beta_0+\beta_1 X_1 + \gamma_{12}X_2 X_1 + \beta_2 X_2 + \varepsilon,$$ and voilá, here we have the product.
