5
$\begingroup$

I have two categorical variables (A and B) in a linear regression. Each is binary.

Both are statistically significant in a model without an interaction term.

When I include the interaction between them, neither the interaction term nor either of the main effects are significant.

However, if I change the reference category of A, B is significant in the model.

As I have read that the main effects in a model that includes an interaction term represent the effect of one variable when the other is 0, and my explanatory variables are binary, I am confused how this does not constitute an interaction. It seems clear that the effect of B depends on A.

Would anyone mind explaining this to me please? I am unsure how to interpret my results.

$\endgroup$
5
$\begingroup$
  • In the model without the interaction, the intercept estimates the response when both A and B are zero. The coefficient for A represents the difference between the response when A is 1, and when A is 0. Likewise, the coefficient for B represents the difference between the response when B is 1, and when B is 0.

  • In the model with the interaction, the intercept again estimates the response when both A and B are zero. The coefficient for A estimates the difference between the response when A=1 and A=0, when B is zero. Likewise, the coefficient for B estimates the difference between the response when B=0 and B=1, when A is zero. The interaction itself is literally the product of A and B, $A \times B$, so this will be zero unless A and B are both 1. This means that the interaction estimates the additional difference for the response between A=0 and A=1, when B changes from zero to 1, or vice versa.

Since the coefficients are estimating different things, the t tests for the coefficients are different tests which obviously will have different results.

Regarding what happens when you change the reference level of A, that is simply a re-parameterisation of the model. The global statistics, such as R squared and the F test will be identical, but the effect on the individual estimates will be different in the two models.

  • In the model with no interaction, the sign of the coefficient of the "new" A will flip, it's standard error will be unchanged (hence the t test will be the same) but the estimate for the intercept will change by exactly the size of the estimate for A (and it's standard error will change, and hence the t test result will change) because it estimates the difference in the response when A is zero which has now changed by exactly what the original estimate of A was.

  • In the model with the interaction, the estimate for the new A and the intercept will change in exactly the same way as for the model with no interaction. The estimate for the interaction should flip signs (with same t test result) but the estimate for B should change by exactly the size of the estimate for the interaction, because it estimates the difference in the response when A is zero, and now A has been reparameterised so of course the standard error and t test result will be different for B

A simple simulation demonstrates all of the above:

> set.seed(15)
> N <- 100
> A <- rbinom(N, 1, 0.4)
> B <- rbinom(N, 1, 0.6)
> Y <- 10 + A + B + 3*A*B + rnorm(N)
> summary(m0 <- lm(Y ~ A + B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   8.5643     0.2721   31.48   <2e-16 ***
A             2.9629     0.2666   11.12   <2e-16 ***
B             2.8035     0.2690   10.42   <2e-16 ***

> summary(m1 <- lm(Y ~ A * B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   9.7563     0.3091  31.566  < 2e-16 ***
A             1.2174     0.3740   3.255  0.00157 ** 
B             1.1116     0.3682   3.019  0.00325 ** 
A:B           2.7988     0.4736   5.910 5.21e-08 ***

# switch the levels of A
> AA <- (A - 1)^2

> summary(m0.1 <- lm(Y ~ AA + B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  11.5271     0.2192   52.60   <2e-16 ***
AA           -2.9629     0.2666  -11.12   <2e-16 ***
B             2.8035     0.2690   10.42   <2e-16 ***

> summary(m1.1 <- lm(Y ~ AA * B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  10.9737     0.2106  52.106  < 2e-16 ***
AA           -1.2174     0.3740  -3.255  0.00157 ** 
B             3.9104     0.2978  13.129  < 2e-16 ***
AA:B         -2.7988     0.4736  -5.910 5.21e-08 ***

> coef(m0.1)[1] - coef(m0)[1] - coef(m0)[2]
(Intercept) 
          0 
> coef(m1.1)[1] - coef(m1)[1] - coef(m1)[2]
(Intercept) 
          0 
> coef(m1.1)[3] - coef(m1)[3] - coef(m1)[4]
          B 
          0 
$\endgroup$
3
  • $\begingroup$ Thank you very much for replying. I understand why the model output is different when I change the reference category. What I don't understand is why I can't interpret these results as showing there is an interaction between my two variables if B only has an effect on A when A = 1 (or vice versa). I realise I am probably being dense, but what am I missing? $\endgroup$
    – Picapica
    Feb 14 '20 at 20:10
  • $\begingroup$ Maybe I could put it this way. If B only affects A when A = 1 and not when A = 0, but the interaction term is not significant, how would you interpret this? According to the reduced model, both A and B have an effect so if I use this information I should interpret it as B equally affecting both A0 and A1. This doesn't seem to be the right thing to do. $\endgroup$
    – Picapica
    Feb 14 '20 at 20:15
  • $\begingroup$ @Picapica sorry for the late reply. Statistical significance is a function of the sample size. It is quite possible that you had insufficient power to detect the effect. Did you do a power analysis prior to collecting data ? $\endgroup$ Feb 19 '20 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.