0
$\begingroup$

I'm looking to use a linear mixed effects model. Each participant will rate a given item using 9 scales, 1 of which is the outcome variable. There are 6 items in total. The aim is to determine which predictor is the strongest.

Due to constraints on the duration of testing sessions, we only want each participant to rate 2 of the 6 items. One option is to arbitrarily divide the 6 items into 3 non-overlapping pairs, and then randomly assign one of these pairs to each participant. Does this count as a nested design? If so, does the following R code look right?

    mod <-lmer(DV~ 1 + P1 + P2 + P3 + P4 + P5 + P6 + P7 + P8 + (1 + P1 + P2 + P3 + P4 + P5 + P6 + P7 + P8|Participant) + (1 + P1 + P2 + P3 + P4 + P5 + P6 + P7 + P8|item-pairings/item), data =mydata)

The other option is to utilize all 15 unique (but overlapping) pairs, and randomly assign one to each participant. Given that the pairs overlap, is this now a crossed design? Should the code look the same as above, but with item no longer specified as nested within item-pairings? What should I do about item-pairings if we go with this design? Enter it as a fixed effect? Theoretically, we're not interested in any effect of item-pairing, but maybe would it be advisable to include it in the model, regardless?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ How many participants are expected? I am thinking that inadvertently we might end up with a crossed design, despite recognising we have a potentially nested design, just because certain items are not replicated enough. $\endgroup$
    – usεr11852
    Commented Apr 23, 2021 at 0:48
  • 1
    $\begingroup$ Based on what is described I would assume that the nesting participant/item is relevant too. And also, so many random slopes... are you sure this won't lead to model identifiability issues? At first instance mod <-lmer(DV~ 1 + P1 + P2 + ... + P8 + (1|Participant/item) + (1 |item_pairings/item), data =mydata) should be adequate for the second design option. $\endgroup$
    – usεr11852
    Commented Apr 23, 2021 at 1:15
  • 1
    $\begingroup$ Finally the first design case seems to me suboptimal as certain participants won't see a particular item (or item-pairing) by design so how will we distinguish from say item 1 being great (getting high scores all-round) and participant 1 being affluent with her scoring (giving high scores all-round) if item 1 is say in the 1-2 item-pairing and participant 1 is within that pairing. The 15 pair option seems much more robust. $\endgroup$
    – usεr11852
    Commented Apr 23, 2021 at 1:17
  • 1
    $\begingroup$ Cool. That's great then. I was thinking you wanted the first design because of some small sample restrictions. In that sense then, the second design would be an obvious choice for me. And right, now that I think of it I understand why participant/item was not picked initially. Yeah, not using probably won't be harmful, if anything it can cause identification issues so dropping it (as you initially did) is probably better. $\endgroup$
    – usεr11852
    Commented Apr 23, 2021 at 1:50
  • 1
    $\begingroup$ No problem at all. I would say yes, if we would think that there might be a framing effect but no, if we think that it won't (be nested). Ultimately the distinction being having "nested" or "crossed" effect would mean distinguishing between using the error structure (1|item_pairings) + (1|item:item_pairings) and (1|item_pairings) + (1|item) respectively. Do note that: (1|item:item_pairings) is the same as (1|item) if item is coded uniquely across item_pairings. Conceptually if we assuming no framing effects, then it's not uniquely coded. $\endgroup$
    – usεr11852
    Commented Apr 23, 2021 at 2:49

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.