Suppose we have two categorical variable: C and B. C has 3 levels(0,1,2) and B has 3 levels(0,1,2). Each participant is exposed to each of these variables under a latin square design such that under each session each participant experiences all the three levels under category C and also all the three levels under category B. I also introduce participant ID as the random effect to account for any individual differences. I fit a mixed effects model against a certain prediction P in the following way:

mixed.model.out = lmer(P ~ C*B + (1|pid), data=data,REML=F)

Luckily, when I ran the above model, I found a significant main effect from C as in, the coefficient of the contrast 0-1 and 0-2 are significant. I used the lmerTest library to check for significances.

But, when I changed the reference level for B from 0 to 1, I lost the significance for the contrast variables in C that I earlier found to be significant. Why is this happening? I also dropped the random effect and fit a simple lm model and I observe a similar trend where I lose significance for the contrasts when I change the baseline for variable B. Unfortunately I can't share my exact numbers but if anyone has run into similar issues I would highly appreciate your feedback.

Here are a few other backup tests I ran to check if the main effects I was expecting are in fact there or not.

mixed.model.null = lmer((1|pid), data=data,REML=F)

mixed.model.B = lmer(B+(1|pid), data=data,REML=F)

mixed.model.C = lmer(C+(1|pid), data=data,REML=F)

mixed.model.BCinter = lmer(B*C+(1|pid), data=data,REML=F)

mixed.model.BCnointer = lmer(B+C+(1|pid), data=data,REML=F)

anova(mixed.model.null,mixed.model.C): Significant

anova(mixed.model.null,mixed.model.B): Not Significant

anova(mixed.model.BCinter,mixed.model.BCnointer): Significant
  • 1
    $\begingroup$ This question is not presented very clearly. $\endgroup$ – Michael R. Chernick Mar 26 '17 at 19:23
  • 3
    $\begingroup$ to answer the question in the title: yes, of course it's possible. The p-values for the individual terms are testing a completely different hypotheses when you change the reference level. $\endgroup$ – gammer Mar 26 '17 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.