1
$\begingroup$

I conducted a randomized crossover study in which I had two within subject factors; condition and Time. When running a repeated measures ANOVA the analysis shows significant main effect of condition both in R and SPSS and the p-values are the same. However, when changing to analyze the data using a mixed effects model with condition and time as fixed effects and subjects as a random effect the p-value differed drastically between R and SPSS.

In fact when removing the interaction (condition*Time) from the model the main effect of condition change to be significant in R and the p-values are almost the same than those in SPSS. however, when including interaction in R the main effect of condition becomes non significant. I just have started using R and I'm wondering whether my model is incorrectly specified.

Here is the code in R: Runing the analysis without condition*Time interaction

library(lme4) # for lmer
library(lmerTest)
library(car)
contrasts(metformin$Condition) <- "contr.sum"
contrasts(metformin$Order)
contrasts(metformin$Time)
contrasts(metformin$Condition) 
m = lmer(Glycémie ~ (Time + Condition) + (1|id), data=metformin,)
summary(m)
Anova(m, type=3, test.statistic="F")

m = lmer(Glycémie ~ (Time + Condition) + (1|id), data=metformin,)
> summary(m)
Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: Glycémie ~ (Time + Condition) + (1 | id)
Data: metformin

REML criterion at convergence: 659.9

Scaled residuals: 
 Min      1Q  Median      3Q     Max 
-2.7000 -0.6104 -0.0516  0.4735  3.9842 

Random effects:
Groups   Name        Variance Std.Dev.
id       (Intercept) 0.4155   0.6446  
Residual             0.5475   0.7400  
Number of obs: 273, groups:  id, 13

Fixed effects:
          Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)   5.11795    0.21448  21.87843  23.862  < 2e-16 ***
Time2        -0.18718    0.16757 252.00000  -1.117  0.26505    
Time3         0.47436    0.16757 252.00000   2.831  0.00502 ** 
Time4         0.92821    0.16757 252.00000   5.539 7.63e-08 ***
Time5         0.66923    0.16757 252.00000   3.994 8.54e-05 ***
Time6        -0.14103    0.16757 252.00000  -0.842  0.40081    
Time7        -0.94103    0.16757 252.00000  -5.616 5.16e-08 ***
Condition1    0.15201    0.06334 252.00000   2.400  0.01711 *  
Condition2    0.10366    0.06334 252.00000   1.637  0.10293    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
        (Intr) Time2  Time3  Time4  Time5  Time6  Time7  Cndtn1
Time2      -0.391                                                 
Time3      -0.391  0.500                                          
Time4      -0.391  0.500  0.500                                   
Time5      -0.391  0.500  0.500  0.500                            
Time6      -0.391  0.500  0.500  0.500  0.500                     
Time7      -0.391  0.500  0.500  0.500  0.500  0.500              
Condition1  0.000  0.000  0.000  0.000  0.000  0.000  0.000       
Condition2  0.000  0.000  0.000  0.000  0.000  0.000  0.000 -0.500
> Anova(m, type=3, test.statistic="F")
Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)

Response: Glycémie
                F Df  Df.res    Pr(>F)    
(Intercept) 569.4116  1  21.878 < 2.2e-16 ***
Time         28.2874  6 252.000 < 2.2e-16 ***
Condition     8.2454  2 252.000 0.0003399 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>

Running the model including condition*Time interaction

m = lmer(Glycémie ~ (Time * Condition) + (1|id), data=metformin,)
> summary(m)
Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: Glycémie ~ (Time * Condition) + (1 | id)
   Data: metformin

REML criterion at convergence: 662.3

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.6704 -0.6568 -0.0335  0.5284  3.4914 

Random effects:
 Groups   Name        Variance Std.Dev.
 id       (Intercept) 0.4160   0.6449  
 Residual             0.5379   0.7334  
Number of obs: 273, groups:  id, 13

Fixed effects:
                  Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)        5.11795    0.21398  21.67450  23.918  < 2e-16 ***
Time2             -0.18718    0.16608 240.00000  -1.127  0.26085    
Time3              0.47436    0.16608 240.00000   2.856  0.00466 ** 
Time4              0.92821    0.16608 240.00000   5.589 6.19e-08 ***
Time5              0.66923    0.16608 240.00000   4.030 7.50e-05 ***
Time6             -0.14103    0.16608 240.00000  -0.849  0.39665    
Time7             -0.94103    0.16608 240.00000  -5.666 4.17e-08 ***
Condition1         0.02051    0.16608 240.00000   0.124  0.90180    
Condition2         0.06667    0.16608 240.00000   0.401  0.68847    
Time2:Condition1   0.09487    0.23487 240.00000   0.404  0.68662    
Time3:Condition1   0.44103    0.23487 240.00000   1.878  0.06163 .  
Time4:Condition1   0.51795    0.23487 240.00000   2.205  0.02839 *  
Time5:Condition1   0.09231    0.23487 240.00000   0.393  0.69466    
Time6:Condition1  -0.14359    0.23487 240.00000  -0.611  0.54154    
Time7:Condition1  -0.08205    0.23487 240.00000  -0.349  0.72714    
Time2:Condition2   0.07179    0.23487 240.00000   0.306  0.76012    
Time3:Condition2  -0.16667    0.23487 240.00000  -0.710  0.47864    
Time4:Condition2  -0.09744    0.23487 240.00000  -0.415  0.67862    
Time5:Condition2   0.23077    0.23487 240.00000   0.983  0.32683    
Time6:Condition2   0.14103    0.23487 240.00000   0.600  0.54878    
Time7:Condition2   0.07949    0.23487 240.00000   0.338  0.73534    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 21 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

> Anova(m, type=3, test.statistic="F")
Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)

Response: Glycémie
                      F Df  Df.res Pr(>F)    
(Intercept)    572.0617  1  21.674 <2e-16 ***
Time            28.7974  6 240.000 <2e-16 ***
Condition        0.1506  2 240.000 0.8602    
Time:Condition   1.3786 12 240.000 0.1767    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Here is the SPSS syntax and the output for type III tests


DATASET ACTIVATE DataSet1.
MIXED Glycémie BY id cond Time sexe Order
  /CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, 
    ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED=cond Time cond*Time | SSTYPE(3)
  /METHOD=REML
  /RANDOM=INTERCEPT id | COVTYPE(VC)
  /EMMEANS=TABLES(cond) COMPARE ADJ(BONFERRONI)
  /EMMEANS=TABLES(Time) COMPARE ADJ(BONFERRONI)
  /EMMEANS=TABLES(cond*Time) .
3 Mixed The final Hessian matrix is not positive definite although all convergence criteria are satisfied. The MIXED procedure continues despite this warning. Validity of subsequent results cannot be ascertained. 

Type III Tests of Fixed Effects a               
Source  Numerator df    Denominator df  F    Sig.
Intercept   1            12,000    806, 086,  000
cond        2              240      8,  394,  000
Time        6              240     28,  797,  000
cond * Time 12             240      1,  379,  177
a Dependent Variable: Glycémie.             


'''
Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Have you read the Warning section of help("Anova")? $\endgroup$
    – Roland
    Commented May 3, 2022 at 5:59
  • $\begingroup$ In particular with respect to the comment from @Roland, although you specified contr.sum for Condition you did not seem to do that for Time. I think you might need to specify orthogonal contrasts for both interacting factors for the Type III test to make sense, even though you seem to have a completely balanced study. $\endgroup$
    – EdM
    Commented May 3, 2022 at 15:28
  • $\begingroup$ Yes, @Roland thank you, please any suggested readings on how to solve this issue ?? $\endgroup$
    – Methnani jabeur
    Commented May 4, 2022 at 11:54
  • $\begingroup$ Thank you @Ed, please any suggested readings on orthoogonal contrasts? $\endgroup$
    – Methnani jabeur
    Commented May 4, 2022 at 11:55
  • $\begingroup$ As the car::Anova help page says, for Type III contrasts to make sense in R with interactions, you normally need to use contr.sum, contr.poly or contr.helmert for coding the categorical predictors involved. You did that for Condition, not for Time. See this page for how those are done. Why do you need a mixed model with type III contrasts and their associated difficulties at all? It seems that you have a completely balanced design so that repeated measures analysis works fine. $\endgroup$
    – EdM
    Commented May 4, 2022 at 12:57

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.