Comparison / Selection of covariance structures - linear mixed model in SPSS

Question

Hi together,

I am currently trying to build a linear mixed model with repeated measurements in SPSS. I would expect that the correlation between my measurements is highest at adjacent time points, so my guess was that AR1 (autoregressive structure) is the right covariance structure in my case. My syntax for this first model (model1) is:

MIXED measurement BY female meadiansplit time WITH age
/CRITERIA=DFMETHOD(SATTERTHWAITE) CIN(95) MXITER(100) MXSTEP(10) SCORING(1)
SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
/FIXED=female mediansplit time age mediansplit*time | SSTYPE(3)
/METHOD=ML
/REPEATED=time | SUBJECT(study_id) COVTYPE(AR1)

As a comparison to this I also calculated a second model (model2) which is exactly the same like model1 but with an unstructured (UN instead of AR1) covariance structure.

(dependent = measurement; factors = female, mediansplit (median split of a scale, coded as 1 for the upper half and 0 for the lower half), time (7 time points); covariate=age in years (used as covariate as it is a continuous variable))

Model1: -2LL = -563, AIC= -527, parameters 18
Model2: -2LL = -701, AIC= -613, parameters 44
Difference -2LL: 138, difference parameters: 26

--> Model2 seems to fit better (p=0,01), although it includes way more parameters and has not the expected covariance structure. Unfortunately, there are also differences in the significance of my fixed effects. While the interaction of median split and time (which is of great importance to me) is significant in model1, it is not in model2.
Which model is the better one in this case? Model1 with less parameters and the expected covariance structure or model2 with more parameters but a better model fit?

Thanks!

Answer 1

It is not possible to give a definitive answer without accesss to the data itself, and even with access to the data it still might not be possible. However, we can say a few things about this situation.

• Given that you expect high autocorrelation, I would suggest sticking with the first model.

• The 2nd model, which has many more parameters, may very well be over-fitting the data, which results in an, apparent, better fit.

• You could explore the previous point by using the same model to make predictions on a new dataset, or by splitting the dataset if you don't have access to new data.