1
$\begingroup$

I have been looking through material on difference-in-difference (DID) models, and I understand the interaction term in a regression model can estimate the DID effect. However, fundamentally for statistical models, the interaction term may only be interpreted if the main effects are statistically significant as well.

Do we also apply the same reasoning and only interpret a DID estimate if the main effects in the model are statistically significant?

$\endgroup$

1 Answer 1

2
$\begingroup$

That is simply untrue, its more a rule of thumb than anything. For example in R.

> x1 <- rnorm(100)
> x2 <- rnorm(100)
> y  <- x1 * x2 + rnorm(100)
> 
> 
> summary(lm(y ~ x1 + x2))

Call:
lm(formula = y ~ x1 + x2)

Residuals:
    Min      1Q  Median      3Q     Max 
-5.4396 -0.9537  0.0591  1.0530  3.1617 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept)  0.06657    0.13804   0.482   0.6307  
x1           0.23257    0.13433   1.731   0.0866 .
x2          -0.21193    0.13720  -1.545   0.1257  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.37 on 97 degrees of freedom
Multiple R-squared:  0.05351,   Adjusted R-squared:  0.034 
F-statistic: 2.742 on 2 and 97 DF,  p-value: 0.06943

> summary(lm(y ~ x1 + x2 + x1 * x2))

Call:
lm(formula = y ~ x1 + x2 + x1 * x2)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.79339 -0.72311  0.01982  0.78272  2.39362 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.05560    0.10767   0.516    0.607    
x1           0.05168    0.10720   0.482    0.631    
x2           0.04472    0.11176   0.400    0.690    
x1:x2        0.98052    0.12309   7.966 3.35e-12 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.069 on 96 degrees of freedom
Multiple R-squared:  0.4302,    Adjusted R-squared:  0.4124 
F-statistic: 24.16 on 3 and 96 DF,  p-value: 9.881e-12  
$\endgroup$
4
  • $\begingroup$ Thanks for your prompt comment - I guess my concern was that while we can fit models, the question is whether the estimates are valid for use. $\endgroup$
    – Sheryl
    Commented Jan 14, 2022 at 8:13
  • $\begingroup$ I fail to understand why you think it is not possible.. $\endgroup$
    – Kozolovska
    Commented Jan 14, 2022 at 9:12
  • $\begingroup$ @Sheryl there can be confusion about "main effect" terminology when there are interactions. See this thread, for example. If by that you mean the coefficients of the individual predictors in a model with interactions, like the second model in this answer, such "main effects" aren't uniquely defined as they depend on the reference values/centering of the predictors with which they interact. See this thread for a worked-through example. $\endgroup$
    – EdM
    Commented Jan 14, 2022 at 17:58
  • $\begingroup$ @Sheryl if by "main effects" you mean the marginal estimates with respect to individual predictors in a model that hasn't incorporated an interaction, like the first model in this answer, then you must recognize that those estimates ignore both the correlations among the predictors that are typically present in observational studies and the potentially important interactions among the predictors. So, either way that you are using the term "main effects," you need to evaluate interaction terms. $\endgroup$
    – EdM
    Commented Jan 14, 2022 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.