2
$\begingroup$

I am trying to apply a linear mixed model on my data. Unfortunately all howto's and papers I found yet do not help me with my design:

I have 30 different texts (stimuli) which are rated on 5 items (on the text quality) by the participants. Every participant rates 6 texts. Text 1 and 2 are rated by ALL participants. The other 28 texts are divided into 7 groups (4 each). The participants are randomly assigned to one of these text-groups (including Text 1 and Text 2 and 4 specific others 1,2,3,4,5,6; 1,2,7,8,9,10 and so on).

I already have the "ideal" ratings as a property of each text for all 30 texts. Now I want to see, if the participants are able to rate them similarly or not.

I therefore want to apply different linear mixed models as follows but I am not sure, if I am allowed to. The participants are, I guess, nested in the texts, but all participants are rating text 1 and 2 so there is no clear cluster. I want to account for text-effects and person effects.

So far, my code for the different models (I want to try all of them and test with an anova which one fits best) is:

#Texts as a random effect

model1 <- lmer(participant_rating ~ perfect_rating + (1|text), data = data)

#plus persons as a random effect

model2 <- lmer(participant_rating ~ perfect_rating + (1|text) + (1|participant), data = data)

#plus slopes

model3 <- lmer(participant_rating ~ perfect_rating + (1 + perfect_rating|text) + (1 + perfect_rating|participant), data = data)

I read in forums that the lme4 package actually has no problem with a lot of different variations in nesting and random effects but I'm afraid I will have to explain why I am allowed to do this.

I am really thankful for any advice :-)

...and I am sorry for the long post..

Edit:

I didn’t know a factor can be nested and not nested (because of texts 1 and 2). And thank you for drawing it. In my head though, it makes more sense that participants (n=300) are nested in the 28 texts. So the 28 texts are the grouping factor? And are models 2 and 3 really still appropriate, I thought I have to specify nested effects: in your example:

#plus persons as a random effect model2 <- lmer(participant_rating ~ perfect_rating + (1|participant/text) + (1|participant), data = data) #plus slopes model3 <- lmer(participant_rating ~ perfect_rating + (1 + perfect_rating|participant/text) + (1 + perfect_rating|participant), data = data)

And in my example:

#plus persons as a random effect model2 <- lmer(participant_rating ~ perfect_rating + (1|text) + (1|text/participant), data = data) #plus slopes model3 <- lmer(participant_rating ~ perfect_rating + (1 + perfect_rating|text) + (1 + perfect_rating|text/participant), data = data)

Thanks for a last advice and I guess then I will be good :-)

$\endgroup$
4
  • $\begingroup$ You are right that lme4 can handle crossed, nested and partially crossed/nested structures without any problem, provided that you code the levels of the variables uniquely. You might find my answer here to be of some use. It doesn't mention partially crossed/nested designs, but it doesn't matter, you don't need to do anything special. $\endgroup$ May 21 '21 at 20:08
  • $\begingroup$ Thank you for your answer. I already read your linked answer before and it was helpful to get to this point. I guess my main problem is that I still can't figure out if and how my variables are nested, which are the levels and if my code is right therefore. I understand the online examples well (the schools or countries and stuff), but my scenario is somehow different and hard to figure out for me at the moment. $\endgroup$
    – HenrikG
    May 21 '21 at 20:47
  • $\begingroup$ OK, I see. Well, in a nutshell, a factor is nested within another factor if it's levels belong to one and only one level of the other one. If it's level's are coded uniquely then you can explore this by a simple cross-tabulation, as shown in that other answer. From your description it seems that text 1 and text 2 are fully crossed, while the others are nested within participants, so model2 or model3 should be appropriate. $\endgroup$ May 21 '21 at 21:45
  • $\begingroup$ You might try drawing a diagram with participants on the top row and texts on the bottom row. Texts 1 and 2 will have an arrow to every particpant (so they are crossed), while all the other texts will have a line to just one participant (so they are nested). That is, if I've understood correctly. Hopefully that will help to make it clearer to you. $\endgroup$ May 21 '21 at 21:47
2
$\begingroup$

As discussed in the comments, either of models 2 or 3 should be appropriate.

To help understand the structure I have drawn a diagram. Sorry that it's done by hand !

enter image description here

It's obviously not possible to show all 30 items, and I don't know how many participants you have, so I can't make it exactly as you describe, but hopefully it makes sense to see where they are crossed and where they are nested.

If you look at the items, items 1 and 2 have lots of lines pointing to them, so they "belong" to many participants. This means they are crossed. Whereas items 3 and onwards "belong" to only 1 participant, so they are nested.

Hope it helps !

Edit: I just noticed I should have drawn a line from participant A to one of the items between 5 and 30 but it doesn't reallty change anything. The main point is the items 1 and 2 are crossed with participants but items 3-30 are not.

Edit: To address the points raised in the Edit to the OP:

In my head though, it makes more sense that participants (n=300) are nested in the 28 texts.

I don't think so. As mentioned, for factor A to be nested within factor B, levels of factor A must uniquely "belong" to levels of factor B - that is, levels of A must belong to one and only one level of factor B. So, referring to my diagram, you can see that the participants all have multiple lines so those levels do NOT uniquely belong to one and only one item/text, whereas if you look at items 3-30, all of them have only 1 line, so they belong to one and only 1 participant. Thus, those items are nested within participants. Now, referring back to your original post:

Text 1 and 2 are rated by ALL participants.

Clearly this means that they are crossed.

The other 28 texts are divided into 7 groups (4 each). The participants are randomly assigned to one of these text-groups (including Text 1 and Text 2 and 4 specific others 1,2,3,4,5,6; 1,2,7,8,9,10 and so on

Now, ignore texts 1 and 2 from this since we have already handled those, so items 3,4,5 and 6 "belong" to participant A, and 7,8,9 and 10 "belong" to particpant B etc. So this satisfies the definition of nesting.

So if we had a fully nested design, you would write (1 | participant/text) as a grouping variable, which is the same as (1 | participant) + (1 | participant:text) (it is literally just shorthand for it. However, since this is partially crossed, and the items (texts) are coded uniquely, what I think you need is ``(1 | participant) + (1 | text)` - ie., model2 or model3 in your original post.

$\endgroup$
2
  • $\begingroup$ Oh wow, thank you very much for that detailed answer which really helped me. My comment was too long so I edited my first question instead (it is at the end). Maybe you could give me a last advice on that :-) $\endgroup$
    – HenrikG
    May 22 '21 at 8:12
  • $\begingroup$ Ah i guess it doesn’t matter as they just have to be coded uniquely like you said.. Thanks a lot :) $\endgroup$
    – HenrikG
    May 22 '21 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.