0
$\begingroup$

I am conducting a linear mixed model with Treatment and Time as fixed effects and Subject as Random Effect. Some subjects are assigned to the Control group, other are assigned to the Treated Group. Every subject is measured in every time point. When running:

anova(lmer(Value~Treatment*Time+(1|Subject),data=df))

I get that: Time is significant, Treatment non-significant and interaction term non-significant. However, in the summary I find that interaction TreatedTime2 is significant. In that case, shouldn't the Anova show that the interaction term is overall significant?
I present a toy example to show the data structure. Individuals are s1,s2,s3,s4 for the Control Group; s5,s6,s7,s8 for the Treated Group.

Treatment t1 t2 t3
Control 10 ; 12 ; 18 ; 20 14 ; 18 ; 22 ; 25 20 ; 22 ; 24 ; 28
Treated 11 ; 14 ; 16 ; 23 20 ; 22 ; 31 ; 38 20 ; 21 ; 26 ; 23

Thank you!

$\endgroup$
0
$\begingroup$

The issue here is probably (as I don't see your output and what exactly you have done, I can't know for sure) multiple testing. If there are, say, two interaction parameters (the issue is more striking if there are more but your example will have just two), say Treatedt2 and Treatedt3, two tests are run for the individual parameters, and that's two chances to find a significant result. Now if you have two chances, finding at least one significant result is easier than finding one if you only run a single test. In fact under null hypothesis, if you test at 0.05-level, for a single test the probability is 0.05, but if you have two tests it is larger (how large exactly depends on whether and how dependent the tests are). The overall test is only a single one, and defined so that under null hypothesis it respects the level, i.e., probability for rejection is 0.05. This means that indeed it can happen that the overall test is not significant whereas one of the individual tests is significant, because the probability for this to happen (note that I'm not talking about any specific individual test, just "one of them, no matter which") is actually larger than 0.05, and is implicitly corrected if you run the overall test that aggregates the information from all the single parameter tests. Some standard advice would be to interpret the single parameter tests only if the overall test is significant (although arguably if the overall test is very borderline like $p=0.051$ one could say that there's some weak indication that something's going on).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.