7
$\begingroup$

I'm not entirely sure of fitting the model for experiment we've made. The variables and relevant description are as follows:

  • ID - participant ID
  • Trial - 60 for each participant
  • Memory - between subject binary factor
  • State - within subject binary factor
  • Correct - whether classification a participant made was correct or not
  • Rating - the judgement made after each trial on four point Likert scale

Procedure brief: each participant (N=60) was randomly assigned to experimental or control group (Memory) and had 120 Trials (60 for State = 0 and 60 for State = 1). Each trial composed of perceptual classification (Correct) and judgment of how easy it was (Rating). The classification problem was randomly selected from two groups each trial (State).

I would like to calculate what impacts the performance (Correct) most - is it memory, state, a specific rating on a scale or any combination of above? I'm not interested in between subject variance, on the oposite, it is a random factor here. Also, it appears that there is bias in responses on Likert scales, so that part of variance should be excluded too.

The way I was thinking to approach this is generalized mixed linear model, but I'm not sure I'm doing it right; there is what I've got so far:

model = glmer(Correct ~ (1|ID/Rating) + Memory * State * Rating, data, family=binomial, 
              control = glmerControl(optimizer="bobyqa", optCtrl = list(maxfun=100000)))

Is this approach correct? I'll appreciate your input.

Relevant resources I used:

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

I don't see why Rating is nested within ID. This means that every every unique Rating belongs to one and only one ID which does not appear to be the case. If Rating is to be treated as random, then these should be crossed random effects. See the answer here for the difference between crossed and nested random effects and how to specify them.

That said, however, Rating should not be specified as a random intercept here because 1) there are only 4 levels, which is insufficient and 2) it is a likert scale item so the assumption of normality of random effects is hardly likely to hold.

So a better model would be

model = glmer(Correct ~ (1|ID) + Memory * State * Rating, data, family=binomial)

Lastly, note that there a several ways that you can treat Rating as an independent variable, see here, here and here for more details.

Improve this answer
$\endgroup$
2
  • $\begingroup$ But: rating is placed as Random nested effect AND as fixed effect. My line of thinking was, that "every unique Rating (that) belongs to one and only one ID" should be removed, and the portion of variance of interest is handled by placing it as fixed effect too. Doesn't it work that way? Thanks. $\endgroup$
    – Piotr Falkowski
    Aug 21, 2016 at 12:27
  • 1
    $\begingroup$ No, it doesn't work that way. The point of including Rating as a random intercept is to control for non-independence of observations within each Rating. Including it as a fixed effect also achieves this, so doing it twice is a kind of over-parameterisation, but not withstanding this, as I mentioned in the two points, you can't include Rating as random anyway. $\endgroup$
    – Robert Long
    Aug 21, 2016 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.