I am curious about how interaction terms are calculated within a multiple linear regression model. I understand that in a simple linear regression, the following holds:

$$y_i=\beta_0+\beta_1x_i+\epsilon_i $$ $$\hat{\beta_1}=\frac{\Sigma(x_i-\bar{x})y_i}{\Sigma(x_i-\bar{x})^2}$$ $$var(\hat{\beta_1})=\frac{\sigma^2}{\Sigma(x_i-\bar{x})^2}$$

Now, if we extend this to a multiple linear regression model, I understand that estimates corresponding to the above are:

$$y_i=\beta_0+\beta_1x_i+\beta_2z_i+\epsilon_i $$

$$\hat{\beta_1}=\frac{\Sigma\hat{r}_{i1}y_i}{\Sigma\hat{r}^2_{i1}}$$ and $$var(\hat{\beta_1})=\frac{\sigma^2}{\Sigma(x_i-\bar{x})^2(1-R^2_1)}$$

where $r^2_1$ are the residuals from a simple regression of $x$ on $z$, and $R^2_1$ is the R-squared from regressing $x$ on $z$. This is following a partialling out interpretation of multiple linear regression.

My question is how the above can be expressed for an interaction term. Say we have the model:

$$y_i=\beta_0+\beta_1x_i+\beta_2z_i+\beta_3xz_i+\epsilon_i $$

Do I substitute the product of $x$ and $z$ into the above equations? This would seem strange, as it would rely upon regressing the product of $x$ and $z$ upon $x$ and $z$ to calculate $R^2_1$. Otherwise is there some aspect of the estimation procedure of which I am unaware? It would be great to know from where these estimates magically appear.

  • $\begingroup$ Strange or not, what you describe is exactly right. It's usually a good idea, when hand calculating, to standardize $x$ and $z$ first, for otherwise $xz$ can be almost collinear with $x$ or $z$. $\endgroup$
    – whuber
    Commented Nov 24, 2017 at 16:33
  • $\begingroup$ That's really interesting. Thank you for clarifying and for the advice on standardising the variables. $\endgroup$
    – Wes
    Commented Nov 24, 2017 at 16:38
  • $\begingroup$ Please see stats.stackexchange.com/search?q=%5Binteraction%5D+compute for additional posts on this topic. $\endgroup$
    – whuber
    Commented Nov 24, 2017 at 16:40


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.