I conducted an experiment with a system that allows to perform a task on a test-object using several methods. The quality of how the task was performed is measured by an error.

I measured the errors of 10 methods each conducted by 5 users on 2 test-objects whereas each user conducted 6 attempts per method and test-objects.

I would like to model this setup using a linear mixed effects model. In particular, I would like to evaluate whether there are (significant) differences

  • between users (intrauser reliability)
  • within users (interuser reliability)
  • across test-objects (across test-object reliability)
  • between methods

I think I have

  • fixed effects: method
  • random effects: user, test-object

Accordingly, the model would be

error ~ methods + (1|user) + (1|test-object)

Is that correct?

Furthermore, how can I know if I have a nested structure? Accordingly I would require a term like (1|User/Attempt)?


  • 1
    $\begingroup$ If I understand correctly, there are only 2 levels of the test-object variable? If so then it is questionable to use test-object as a random effect - just retain it as a fixed effect $\endgroup$ – Robert Long Jul 8 '18 at 17:04
  • $\begingroup$ Yes that is correct, I can change it to a fixed effect. The question if I need a term like (1|User/Attempt) remains though? $\endgroup$ – user137589 Jul 9 '18 at 14:54

From your description, to fit a linear mixed effects model to your data you could use:

error ~ methods + (1|user) + (1|test-object)

Nesting is a property of the experimental design itself. From your description, a test-object does not belong to a particular user, and neither does a particular user belong to a particular test-object in the sense that a child would belong to, say, a class in a school, in a nested design. So you do not have nested data.

Therefore you do not need (1|user/test-object) - though it is worth noting that the results should be the same, provided that the variables are encoded appropriately. See the accepted answer here for more details.

Finally, with only 2 levels of test-object, it doesn't really meet the requirements for random effects, so a better model may be:

error ~ methods * test-object + (1|user)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.