5
$\begingroup$

When I was dealing with my data, I got the p-value for my interaction term as 0.143, which in hypothesis testing, we cannot reject the null that there are no effects caused by the interaction.

In this case, the interaction plot should be more parallel as the interaction is not significant.

However, when I plot the interaction plot for my data, this came out: enter image description here

This doesn't seem to be parallel at all, which confuses me given such high p-value.

What am I missing here? Thanks!

$\endgroup$
2
  • $\begingroup$ As Robert mentions, sample size is the likely culprit. How many samples do you have? $\endgroup$
    – Dave
    Feb 14, 2021 at 13:35
  • $\begingroup$ @Dave There were like 48 samples, so I think the sample size is too small $\endgroup$
    – Edi
    Feb 20, 2021 at 15:42

1 Answer 1

4
$\begingroup$

One possible explanation here is that you don't have sufficient statistical power to detect a "significant" interaction effect.

We can do a simple simulation to show this. We simulate data similar to that in the question:

set.seed(48)
dt <- expand.grid(car = c("A", "B", "C"), gas = c("petrol", "unleaded", "diesel", "biofuel"), reps = 1:2)
X <- model.matrix( ~ car * gas, dt)

betas <- c(25, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 2)
dt$miles <- X %*% betas + rnorm(nrow(dt))

So we have simulated data where we expect an interaction carC:gasbiofuel but no other interaction, similar to the OP

ggplot(dt, aes(y = miles, x = car, group = gas, color = gas)) + geom_smooth(method='lm', formula= y~x, se = FALSE)

enter image description here

However, if we run an ANOVA we find:

summary(aov(miles ~ car * gas, dt))
##             Df Sum Sq Mean Sq F value Pr(>F)  
## car          2 14.852   7.426   3.718 0.0554 .
## gas          3  8.947   2.982   1.493 0.2663  
## car:gas      6 12.189   2.032   1.017 0.4590  

and we obtain a non-significant interaction term.

However if we increase the sample size:

dt <- expand.grid(car = c("A", "B", "C"), gas = c("petrol", "unleaded", "diesel", "biofuel"), reps = 1:10)
X <- model.matrix( ~ car * gas, dt)
dt$miles <- X %*% betas + rnorm(nrow(dt))

summary(aov(miles ~ car * gas, dt))

##              Df Sum Sq Mean Sq F value   Pr(>F)    
## car           2  45.96  22.981  24.213 2.05e-09 ***
## gas           3  22.72   7.572   7.978 7.47e-05 ***
## car:gas       6  12.57   2.095   2.207   0.0477 *  
## Residuals   108 102.50   0.949       

and we obtain a significant interaction term.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.