Suppose we have two categorical variable: C and B. C has 3
levels(0,1,2) and B has 3
levels(0,1,2). Each participant is exposed to each of these variables under a latin square design such that under each session each participant experiences all the three levels under category C and also all the three levels under category B. I also introduce participant ID as the random effect to account for any individual differences. I fit a mixed effects model against a certain prediction P in the following way:
mixed.model.out = lmer(P ~ C*B + (1|pid), data=data,REML=F)
Luckily, when I ran the above model, I found a significant main effect from C as in, the coefficient of the contrast 0-1 and 0-2 are significant. I used the
lmerTest library to check for significances.
But, when I changed the reference level for B from 0 to 1, I lost the significance for the contrast variables in C that I earlier found to be significant. Why is this happening? I also dropped the random effect and fit a simple lm model and I observe a similar trend where I lose significance for the contrasts when I change the baseline for variable B. Unfortunately I can't share my exact numbers but if anyone has run into similar issues I would highly appreciate your feedback.
Here are a few other backup tests I ran to check if the main effects I was expecting are in fact there or not.
mixed.model.null = lmer((1|pid), data=data,REML=F) mixed.model.B = lmer(B+(1|pid), data=data,REML=F) mixed.model.C = lmer(C+(1|pid), data=data,REML=F) mixed.model.BCinter = lmer(B*C+(1|pid), data=data,REML=F) mixed.model.BCnointer = lmer(B+C+(1|pid), data=data,REML=F) anova(mixed.model.null,mixed.model.C): Significant anova(mixed.model.null,mixed.model.B): Not Significant anova(mixed.model.BCinter,mixed.model.BCnointer): Significant