SPSS mixed-effects model vs Matlab FITLME : inconstistent results in estimated coefficients

Question

I am trying to double-check the results I am getting with the matlab fitlme function by comparing it with the output of the SPSS MIXED procedure. The results of the F tests match perfectly, while the estimated coefficients vary quite a lot. In particular, for one instance they yield opposite signs. Here's the settings:

I have 77 subjects, I have 1 continuous DV (activation), 2 continuous IVs (score1 and score2) and 1 categorical IV (condition) with 2 levels. Each subject undergoes both conditions.

In matlab, I code the model as:

formula= 'activation ~  condition * score1 + condition * score2 + (condition|subject)';
lmeO= fitlme(ds, formula, 'FitMethod', 'ML', 'DummyVarCoding','effects','CovariancePattern','Isotropic');
ss=anova(lme,'DFMethod','satterthwaite'); 

And this is what I get:

Model information:
    Number of observations             154
    Fixed effects coefficients           6
    Random effects coefficients        154
    Covariance parameters                2

Formula:
    beta ~ 1 + condition*score2 + condition*score1 + (1 + condition | subject)

Model fit statistics:
    AIC        BIC        LogLikelihood    Deviance
    -1653.7    -1629.4    834.85           -1669.7 

Fixed effects coefficients (95% CIs):
    Name                         Estimate       SE            tStat       DF     pValue        Lower          Upper      
    '(Intercept)'                 0.00035541    0.00036932     0.96233    148       0.33745    -0.00037442      0.0010852
    'condition_0'                 0.0013595    0.00036932       3.681    148    0.00032481     0.00062966      0.0020893
    'score2'                      2.3196e-06    6.8395e-06     0.33915    148       0.73498    -1.1196e-05     1.5835e-05
    'score1'                     -3.841e-06    3.8722e-06    -0.99193    148       0.32285    -1.1493e-05      3.811e-06
    'condition_0:score2'          7.313e-06    6.8395e-06      1.0692    148        0.2867    -6.2026e-06     2.0829e-05
    'condition_0:score1'         -1.2532e-05    3.8722e-06     -3.2365    148     0.0014931    -2.0184e-05    -4.8805e-06

K>> ss=anova(lmeO,'DFMethod','satterthwaite')

ss = 


    ANOVA marginal tests: DFMethod = 'Satterthwaite'

    Term                       FStat      DF1    DF2        pValue    
    '(Intercept)'              0.92609    1      154       0.33739
    'condition'                13.55      1      154    0.00032081
    'score2'                   0.11502    1      154       0.73496
    'score1'                   0.98393    1      154       0.32279
    'condition:score2'         1.1433     1      154       0.28664
    'condition:score1'         10.475     1      154     0.0014814

When I try to do exactly the same in SPSS, I code it this way:

MIXED activation BY Condition WITH score1 score2
  /CRITERIA=CIN(95) MXITER(1000) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0,
    ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
  /FIXED=Condition score1 score2 Condition*score1 Condition*score2 | SSTYPE(3)
  /METHOD=ML
  /PRINT=DESCRIPTIVES G  SOLUTION TESTCOV
  /REPEATED=Condition | SUBJECT(subject) COVTYPE(ID)
  /EMMEANS=TABLES(Condition) COMPARE ADJ(BONFERRONI).

And this is what I get

    Type III Tests of Fixed Effects             
    Source      Numerator   df    Denominator df    F        Sig.
    Intercept           1         154               .926    .337
    Condition           1         154               13.550  .000
    score1              1         154               .984    .323
    score2              1         154               .115    .735
    Condition * score1  1         154               10.475  .001
    Condition * score2  1         154               1.143   .287

a Dependent Variable: activation    

So, same results from the ANOVA. But the estimated coefficients are quite different:

    Estimates of Fixed Effectsa                         
    Parameter               Estimate    Std. Error      df         t        Sig.    95% Confidence Interval 
    Intercept               -.001004     .000522        154.000   -1.922    .056    -.002036    2.772913E-5
    [Condition=0]           .002719      .000739        154.000    3.681    .000    .001260        .004178
    [Condition=1]   0b  0   
    score1                  8.691529E-6  5.476117E-6    154        1.587    .115    -2.126475E-6    1.950953E-5
    score2                  -4.993413E-6 9.672506E-6    154.000    -.516    .606    -2.410133E-5    1.411451E-5
    [Condition=0] * score1  -2.506497E-5 7.744399E-6    154.000    -3.237   .001    -4.036394E-5    -9.766007E-6
    [Condition=1] * score1  0b  0   .   .   .   .   .
    [Condition=0] * score2  1.462609E-5  1.367899E-5    154.000     1.069   .287    -1.239659E-5    4.164877E-5
    [Condition=1] * score2  0b  0   .   .   .   .   .

    a Dependent Variable: HbO_Beta.                         
    b This parameter is set to zero because it is redundant.                            

Curiously, when I used the "reference" Dummy coding in matlab, the estimates of interactions agree, although opposite in sign, but that's because it's using th two different levels of condition, but still I get very different results on the main effects of score1 and score2:

K>> lme= fitlme(ds, formula, 'FitMethod', 'ML', 'DummyVarCoding','reference','CovariancePattern','Isotropic')

lme = 


Linear mixed-effects model fit by ML

Model information:
    Number of observations             154
    Fixed effects coefficients           6
    Random effects coefficients        154
    Covariance parameters                2

Formula:
    activation ~ 1 + condition*score2 + condition*score1 + (1 + condition | subject)

Model fit statistics:
    AIC        BIC        LogLikelihood    Deviance
    -1653.7    -1629.4    834.85           -1669.7 

Fixed effects coefficients (95% CIs):
    Name                         Estimate       SE            tStat      DF     pValue       Lower          Upper     
    '(Intercept)'                   0.0017149     0.0005223     3.2833    148    0.0012802     0.00068277      0.002747
    'condition_1'                  -0.002719    0.00073865     -3.681    148    0.0003248     -0.0041786    -0.0012593
    'score2'                        9.6327e-06    9.6725e-06    0.99588    148      0.32093    -9.4814e-06    2.8747e-05
    'score1'                       -1.6373e-05    5.4761e-06      -2.99    148    0.0032685    -2.7195e-05    -5.552e-06
    'condition_1:score2'           -1.4626e-05    1.3679e-05    -1.0692    148       0.2867    -4.1657e-05    1.2405e-05
    'condition_1:score1'           2.5065e-05    7.7444e-06     3.2365    148    0.0014931     9.7611e-06    4.0369e-05

Can anyone help me shed some light on this? How are MIXED and FITLME different in estimating the coefficients?

Thanks a lot in advance

Answer 1

Based on the MATLAB documentation, I'd say that if you specified the full option on DummyVarCoding your results would match those from SPSS MIXED, which uses what is sometimes called full indicator parameterization for factors (one indicator or dummy for each level of the factor) and a generalized inverse that has the effect of aliasing to 0 parameters associated with redundant levels of factors.

A model with an interaction between two predictors means that the effect of each of these predictors is conditional on the level of the other predictor. The "main" effects estimates are dependent on the coding of factors, as is the intercept.

Answer 2

This answer about similar differences between R and SPSS regression reports should shed some light on your problem. To compare so-called "main effects" among software implementations, particularly in models that involve interaction terms, you really need to know how the reference values of predictors are coded by the software.

For example, with treatment coding (default in R and SPSS, don't know about MATLAB), a reported individual regression coefficient ("main effect") represents the difference associated with that individual predictor when all the other predictors are at 0 (continuous predictors) or at their reference levels (categorical predictors). But R and SPSS make different choices for the reference level of a categorical predictor: first versus last, respectively.

So there can always be issues of this type with categorical predictors, at least in terms of signs of reported regression coefficients. When there are more than 2 levels of a categorical predictor, even the values of reported coefficients will differ depending on the choice of its reference level.

Applying this principle to cases with interactions, as @DavidNichols notes in another answer, the "main effect" reported for a predictor involved in an interaction will necessarily depend on the coding of the predictor with which it interacts. That's even true for interaction with a continuous predictor; using raw versus mean-centered values for a continuos predictor will change the "main effect" coefficient for any predictor it interacts with even though the model is functionally the same.

There is no good definition of a "main effect" when a predictor is involved in an interaction term,as the point of an interaction is that the effect of one predictor depends on the value of another one. That's why you see us both putting "main" in quotes.