I am curious about how interaction terms are calculated within a multiple linear regression model. I understand that in a simple linear regression, the following holds:

$$y_i=\beta_0+\beta_1x_i+\epsilon_i $$ $$\hat{\beta_1}=\frac{\Sigma(x_i-\bar{x})y_i}{\Sigma(x_i-\bar{x})^2}$$ $$var(\hat{\beta_1})=\frac{\sigma^2}{\Sigma(x_i-\bar{x})^2}$$

Now, if we extend this to a multiple linear regression model, I understand that estimates corresponding to the above are:

$$y_i=\beta_0+\beta_1x_i+\beta_2z_i+\epsilon_i $$

$$\hat{\beta_1}=\frac{\Sigma\hat{r}_{i1}y_i}{\Sigma\hat{r}^2_{i1}}$$ and $$var(\hat{\beta_1})=\frac{\sigma^2}{\Sigma(x_i-\bar{x})^2(1-R^2_1)}$$

where $r^2_1$ are the residuals from a simple regression of $x$ on $z$, and $R^2_1$ is the R-squared from regressing $x$ on $z$. This is following a partialling out interpretation of multiple linear regression.

My question is how the above can be expressed for an interaction term. Say we have the model:

$$y_i=\beta_0+\beta_1x_i+\beta_2z_i+\beta_3xz_i+\epsilon_i $$

Do I substitute the product of $x$ and $z$ into the above equations? This would seem strange, as it would rely upon regressing the product of $x$ and $z$ upon $x$ and $z$ to calculate $R^2_1$. Otherwise is there some aspect of the estimation procedure of which I am unaware? It would be great to know from where these estimates magically appear.

  • $\begingroup$ Strange or not, what you describe is exactly right. It's usually a good idea, when hand calculating, to standardize $x$ and $z$ first, for otherwise $xz$ can be almost collinear with $x$ or $z$. $\endgroup$ – whuber Nov 24 '17 at 16:33
  • $\begingroup$ That's really interesting. Thank you for clarifying and for the advice on standardising the variables. $\endgroup$ – Wes Nov 24 '17 at 16:38
  • $\begingroup$ Please see stats.stackexchange.com/search?q=%5Binteraction%5D+compute for additional posts on this topic. $\endgroup$ – whuber Nov 24 '17 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.