Interpretting a linear regression model with interactions

Question

I'm running a linear regression model to understand the affect of my treatment (reminders) on my outcome variable (Class attendance).

I ran my model with two dummy covariates (age and previously attended class) along with the interaction of these dummies with treatment. The table below shows the results of the regression

If I were to interpret model 3, would it be accurate to think that while the treatment was effective in increasing attendance, this main effect is diminished somewhat when considering the positive and significant interaction the treatment had with individuals who previously attended the class? How do I interpret the second, non-significant interaction term in my regression and comparing it to the main effect for treatment?

Thanks you!

Answer 1

It's risky to ignore an interaction with a substantial magnitude even if it doesn't pass a test of "statistical significance." The magnitude of your point estimate of the older:treatment interaction term is 5 times larger than that of the "significant" previous_attended:treatment interaction. It's twice the magnitude of the treatment single-predictor coefficient.

Once you have a model, it's unwise to remove a term just because it doesn't pass an arbitrary threshold of "statistical significance"; that's to a great extent just a function of sample size. See Frank Harrell's online notes, for example Section 4.12: "Don't remove insignificant individual effects from the model."

With the interactions you should be very wary of interpreting what you seem to consider the "main effect" coefficients (the first 3 rows of the table). There is no single "main effect" for a predictor involved in an interaction, as its association with outcome depends on the values of its interacting predictors. The estimate of its "main effect" and the corresponding p-value depends on the coding of the predictors with which it interacts. See this page for example.

When you have interaction terms, it's safest not to try to interpret the single-predictor coefficients on their own. Evaluate the overall significance of each predictor, including all of its interactions, via ANOVA. Then display the model predictions for representative combinations of predictor values.