I have a data set including four columns. Group variable includes a control group, and a treatment group, measured at 6 times (time0,time1,time2,time3,time4,time5). In total, the data sets include 12 subjects (6 subjects in the control group and 6 subjects in the treatment group).

Subject  Time  Group   Analyte
1         0     1
1         1     1
1         2     1
1         3     1
1         4     1
1         5     1
2         0     1
2         1     1
2         2     1
2         3     1
2         4     1
2         5     1
3         0     1
3         1     1
3         2     1


As I have a small data set, which one of the following function should be used? Especially, Do you recommend to use the model 4 (with an unstructured correlation matrix and weighted)? Why?

The first model is:

aov (Analyte ~ Group*Time + Error(Subject/Time), data=data)

The second model is:

lmer(Analyte~Time + Group + Time:Group + (1|Subject))


The Third model is:

lmer(Analyte~Time + Group + Time:Group + (Group|Subject)) # with different slopes


The fourth model is:

 gls(Analyte ~ time * group, data = data, correlation = corSymm(form = ~ 1 | subject), weights = varIdent(form = ~ 1 | time))

• Whether a random slope or intercept is more appropriate depends on the research question and your prior knowledge. I doubt model 3 could work on a limited dataset though, as (Group | Subject) estimates both. (0 + Group | Subject) is only random slopes. – Frans Rodenburg Aug 31 '18 at 1:04
• @ Frans Rodenburg, thanks for the comment, what is your opinion about model one, aov function? How about gls function? – Farid Aug 31 '18 at 1:53
• To be honest I would prefer a simple correlation structure like compound symmetry (which you essentially imply in models 2 and 3), because that leaves fewer parameters to be estimated. As for the anova version, I know there are some issues with the type III error, so I personally avoid it, but you could check out some Q&As by Ben Bolker (who wrote lme4) or amoeba (who is more familiar with the differences). I believe there is a question outlining all the different ways to model mixed effects. I'll link it if I can find it. – Frans Rodenburg Aug 31 '18 at 2:58
• Didn't find exactly what I had in mind, but related are: (about gls vs lmer) stats.stackexchange.com/a/14185/176202 and (about RM-ANOVA vs mixed models) stats.stackexchange.com/q/24314/176202 – Frans Rodenburg Aug 31 '18 at 7:46