Recently, I have done a fairly complex experiment, and I am having trouble coming up with a model that is suitable for the data. I have spent a few days reading about, e.g., when random effects should be nested or crossed, and which variables should be included in a full model. Yet, the literature that is readable for a non-statistician like myself is usually limited to two nesting levels, whereas I (may) have more. The literature that does seem to apply, is too complex for me to understand. I hope you can help me.

My aim is to specify a full model, and start model simplification from there.

The experiment I have done, was as follows:

  • All participants completed three tasks, in a fixed order (Task1, Task2, Task3).
  • Task 1 consisted of 30 trials, Tasks 2 and 3 both consisted of 10 trials.
  • Each task was completed three times by each participant (Round1, Round2, Round3).
  • The manipulation consisted of one factor with three levels (Condition1, Condition2, and Condition3).
  • The conditions were tied to the rounds, so each participant completed Task 1 in each condition (e.g., 30 trials in Condition1, followed by 30 trials in Condition2, followed by 30 trials in Condition3), before moving on to Task 2.
  • I used six different stimuli to manipulate these conditions (two for each condition; Stimulus1.1, Stimulus1.2, Stimulus2.1 .. Stimulus3.2)
  • The dependent variable is binary

To complicate matters even more, we used 6 different orders of presenting the stimuli.

I think it's easiest to demonstrate the data structure using an example:

order <- rbind(
c('Stimulus1.1', 'Stimulus2.1', 'Stimulus3.1', 'Stimulus2.2', 'Stimulus3.2', 'Stimulus1.2', 'Stimulus3.1', 'Stimulus1.2', 'Stimulus2.1'),
c('Stimulus1.1', 'Stimulus3.1', 'Stimulus2.1', 'Stimulus3.2', 'Stimulus2.2', 'Stimulus1.2', 'Stimulus2.1', 'Stimulus1.2', 'Stimulus3.1'),
c('Stimulus2.1', 'Stimulus3.1', 'Stimulus1.1', 'Stimulus3.2', 'Stimulus1.2', 'Stimulus2.2', 'Stimulus1.2', 'Stimulus2.1', 'Stimulus3.1'),
c('Stimulus2.1', 'Stimulus1.1', 'Stimulus3.1', 'Stimulus1.2', 'Stimulus3.2', 'Stimulus2.2', 'Stimulus3.2', 'Stimulus2.1', 'Stimulus1.1'),
c('Stimulus3.1', 'Stimulus2.1', 'Stimulus1.1', 'Stimulus2.2', 'Stimulus1.2', 'Stimulus3.2', 'Stimulus1.1', 'Stimulus3.2', 'Stimulus2.1'),
c('Stimulus3.1', 'Stimulus1.1', 'Stimulus2.1', 'Stimulus1.2', 'Stimulus2.2', 'Stimulus3.2', 'Stimulus2.1', 'Stimulus3.2', 'Stimulus1.1'))

test <- expand.grid(trial=1:30, task=c('Task1', 'Task2', 'Task3'), round=c('Round1', 'Round2', 'Round3'), pp=1:12)
test <- test[! (with(test, trial > 10 & task == 'Task2')),]
test <- test[! (with(test, trial > 10 & task == 'Task3')),]
test$taskround <- factor(paste(test$task, test$round, sep=':'))
test$task <- factor(test$task)
test$round <- factor(test$round)
test$stimulus <- factor(unlist(lapply(1:nrow(test), function(x) {order[1 + (test[x, 'pp'] %% 6), as.numeric(test[x,'taskround'])]})))
test$condition <- factor(paste('Condition', substr(as.character(test$stimulus), 9,9), sep=''))
test$response <- factor(rbinom(nrow(test),1, prob=.95))
test$pp <- factor(test$pp)

The resulting data structure is:

   trial  task  round pp    taskround    stimulus  condition response
1      1 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
2      2 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
3      3 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
4      4 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
5      5 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
6      6 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
7      7 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
8      8 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
9      9 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1
10    10 Task1 Round1  1 Task1:Round1 Stimulus1.1 Condition1        1

I'm mainly interested in the main effect of Condition.

I am anticipating that the response tendency may differ between tasks and participants. This would translate into a different intercept for each task and participant, correct?

I am also anticipating that the manipulation may affect some participants more than others; and I suspect that different participants may approach different tasks in a different way (e.g., Task 1 may elicit more successes that Task 2, but participant 1 may be more sensitive to this task aspect than participant 2).

My current model is:

glmer(response ~ condition + (1 | pp/task/round) + (0 + task + condition | pp), data=test, family=binomial)

However, I am not sure if this model is correct. My questions are:

  1. I am uncertain about the nesting of the random effects in the first clause. I believe task/round is correct, but I am not sure whether I should think of these as being nested under participants, or crossed with participants as follows: glmer(response ~ condition + (1 | task/round) + (1 | pp) + (0 + task + condition | pp), data=test, family=binomial).

  2. I am not sure at all about the (0 + task + condition | pp) random effects. I get correlations between the task and condition random effects, and I am not sure whether I would want/need these.

  3. I removed the intercept here, because it was already included in the first random effects clause, but I'm not sure about that either.

  4. Whether this model takes into account any order effects, or whether I should explicitly model those?

  5. Whether I should take into account differences between stimuli (and/or the possibility that different participants react differently to different stimuli), and how I should model these (crossed, or nested)?

  6. I'm worried about fatigue effect (lineair or quadratic) over the course of the experiment AND within tasks. I would like to include these effects into the model as well, but again, I'm not sure how to do that.

So, in summary, I have lots of questions about analyzing these data. I think I am mainly interested in recommendations on how to approach this dataset, and suggestions for a suitable full model. Any other tips/suggestions are very welcome as well.

  • $\begingroup$ It is unclear to me whether round and condition actually represent exactly the same thing, e.g. condition1 is always applied on round1, or they are independent, e.g. condition 1 was sometimes applied on round1, sometimes on round2 and sometimes on round3. Could you precise? $\endgroup$ – Charlotte R Sep 20 '16 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.