Mixed Effects Nesting

Question

I have a simple model with mixed effects. I asked subjects ten questions, five easy and five hard, and saw how much they relied on advice, based on who their advisor was (algorithm or peer) and how hard the question was (easy or hard). The model is predicting how much an individual responds to advice. The simplified model is below. AmountOfAdvice is a percentage, between 0 and 100. AdviceSource is a dummy, 0 or 1. Difficulty is a dummy, 0 (easy), 1 (hard). The R code is below.

model11 <- lmer(WOA ~ AlgoGroup*difficulty + (1| ResponseId), data=df)

I want to account for question-level variations as well. If I simply add a Question factor, like in the model below, where question is a factor with ten levels, then I believe I run into a problem because all questions are either easy or hard. E.g. Question 1, 3, 5, 7, and 9 are always easy. Thus, this means my matrix no longer has an inverse. What is the best solution?

    model11 <- lmer(WOA ~ AlgoGroup*difficulty + Question + (1| ResponseId), data=df)

Answer 1

AmountOfAdvice is a percentage, between 0 and 100. AdviceSource is a dummy, 0 or 1

This doesn't seem to correspond to the variables in the models. I don't see AmountOfAdvice or AdviceSource in either of your models. Notwithstanding that, this seems to be a crossed design, not nested. All subjects answered the same 10 questions. There are repeated measures in subjects and also in question, but a particular question does not belong to a particular subject, nor vice versa. You could fit fixed effects for QuestionID but with 10 of them, and them being plausibly samples from a wider population of questions, they are more naturally modelled as random effects.

Your model:

model11 <- lmer(WOA ~ AlgoGroup*difficulty + (1| ResponseId), data=df)

does not make sense if ResponseId is the unit-level measurement. That is, if it is the ID for a particular answer, because there are no repeated measures at that level. The repeated measures are within subjects and questions (crossed).

So an appropriate model would be:

lmer(WOA ~ AlgoGroup * difficulty + (1|QuestionID) + (1| SubjectID), data=df)