Mixed model for learning data?

Question

I’m working with data from a learning experiment in birds and I have some doubts I hope you can help me clarify. I'm interested in comparing the performance in a learning task between male and female birds. We measured the percentage of correct responses for 6 females and 5 males along 6 different days (=sessions), i.e. we have repeated measures along time for each subject. The data looks like this (the percentage of correct responses for each session was arcsin transformed to normalize data = resp):

Ind     sex    session  resp
1       F      1        1.284
1       F      2        1.318
1       F      3        1.231
1       F      4        1.209
1       F      5        1.150
1       F      6        1.571
2       F      1        1.571
2       F      2        1.571
2       F      3        0.955
2       F      4        0.685
2       F      5        1.571
2       F      6        1.130
3       M      1        1.384
3       M      2        1.571
3       M      3        1.231
..      ..     ..        ..
..      ..     ..        ..

So, the question I’m trying to answer is if there are differences in learning between males and females. I was planning to use a mixed model to evaluate differences between sexes, where:

Dependent variable: percentage of correct responses for each session (arcsin transformed to normalize data) = resp

Fixed effects: sex, session and their interaction

Random effect: subject ID

This is my code:

sex<-as.factor(retcolor$sex)
ind<-as.factor(retcolor$ind)
session<-as.factor(retcolor$session)
resp<-as.numeric(retcolor$resp)

Model.1<-lme(resp~sex*session, data=retcolor, random=~1|ind, method="REML")
summary(Model.1)

Linear mixed-effects model fit by REML
 Data: retcolor 
       AIC          BIC         logLik
       26.20109     38.9639     -7.100546

Random effects:
 Formula: ~1 | ind
        (Intercept)  Residual
StdDev:  0.03248164 0.2364255

Fixed effects: resp ~ sex * session 
                Value       Std.Error   DF   t-value    p-value
(Intercept)     1.5339778   0.09082871  53  16.888688   0.0000
sexM            -0.0336711  0.13472075  9   -0.249933   0.8083
session         -0.0549619  0.02307276  53  -2.382112   0.0208
sexM:session    -0.0256210  0.03422244  53  -0.748659   0.4574

Correlation: 
                 (Intr)    sexM    sessin
sexM            -0.674              
session         -0.889     0.599       
sexM:session     0.599    -0.889   -0.674

Standardized Within-Group Residuals:
        Min          Q1         Med          Q3         Max 
-2.62037068 -0.57627675 -0.07498751  0.65682708  2.00819167 

Number of Observations: 66
Number of Groups: 11


anova(Model.1)

                 numDF  denDF   F-value     p-value
(Intercept)             1        53     1752.8207   <.0001
sex                     1         9     4.0007      0.0765
session                 1        53     15.2789     0.0003
sex:session             1        53     0.5605      0.4574

Do you think I’m going in the right direction?

My main doubt is about the random effect: is it OK like I wrote it?: random=~1|ind.

This means this a random intercept model, right? Or should I use a random slopes model? (this is where I get lost).

Answer 1

In the random effects model, I would consider using random slopes: otherwise, you are implying that each bird in each group learns at the exact same rate.

But just a word of advice: I analyze behavioral data like this quite often, and I've grown to dislike standard mixed effects models for this type of data. The reason for this data will show many aspects that are problematic for simple linear mixed effects model. Such as: the real learning effect is very non-linear: there's usually a large learning effect early on which then flatlines. Secondly, as subjects learn more, the variance of time until success reduces substantially (in fact, it makes sense to treat this as time to event data rather than simple linear regression). Thirdly, there's an annoying slope-intercept interaction; subjects who preform very well early on typically show very little improvement. So if you're just comparing simple slopes, subjects who "get it" on their first attempt don't improve much at all, showing little learning effect. This is logical, but not what you're really interested in the study; I don't think you want to be reporting that groups who "got it" right away were learning slower than those who didn't get it all early on but had lots of room for improvement.

So there's various directions you can take this. You can make a really complicated mixed effects model. But with only 6 observations per subject, I'm not such a fan.

Or, if your question of interest is really simple (overall, how do males compare to females) and that you have a nice balanced design (equal measurements per subject), you can make it much simpler.

The simple model I personally like is at each level (i.e. trial 1, 2, ...), replace the scores with their ranks (so subject who finished first = 1, second = 2, etc.). Then, for each subject, compute a (potentially weighted) mean rank score. Now we can do a simple t-test, linear regression, etc. on this mean rank score (one per individual, each one is independent).

The reason I prefer using the ranks over the raw scores is this tackles the issue that there is higher variance on the earlier scores than the later scores. Otherwise, the mean scores can be completely dominated by the first score, which is not what we are interested in. Moreover, if you're interested in a metric that's more reflective of ability after learning, you can take a weighted average that places heavier weight on later scores; I find that linear ranks (weight of first = 1, weight of second = 2, etc.) work well in the datasets I've seen.

Based on power simulations I've run, even if the model is simulated exactly according the mixed model scheme, this ranked metric results in very little power loss: if the mixed effects model has power = 0.5, then the mean rank statistic will have power, say, 0.48. And once you start simulating data that deviates from the simple model (i.e. declining learning effect), the mean rank statistic displays much, much higher power.

Of course, the mean ranks doesn't really give individual trajectories. But I'm not so sure you're that concerned with this; in the studies I've seen, the researcher typically is mostly just interested in showing there is a difference between groups.

Answer 2

Considering your question:

So, the question I’m trying to answer is if there are differences in learning between males and females.

it will be simpler to find mean for each individual for all sessions. Then do t-test for resp by sex.