I am running the following study, and I am unsure about the best analytical approach. I have two ideas, but I'm not sure either one is correct. I am grateful for any feedback.
I am conducting a study to understand better how people use different environmental features to learn specific paths. I plan to have participants navigate a novel environment ten times. The environment contains three types of features that are next to decision points (i.e., junctions where they must choose a direction to travel). No choice point will have more than one feature. There will be five choice points for each type of feature. We will measure errors at decision points.
My main research question is: does the type of feature affect learning over repeated trials? In particular, I hypothesize that one type of feature better facilitates learning. Thus, I anticipate fewer errors at those choice points, either from the very beginning or a larger (negative) slope.
My plan was to use a multilevel model. I had two possible models I was thinking about using. I plan to use lmer() in R.
In the two-level model, trial would be at level one and participant at level two. Each trial would consist of three observations (i.e., number of errors at the choice points containing one of the three features). The model would like this:
two.level.model <- lmer(errors ~ trial * feature + (trial | participant)
The second option I was thinking about stems from reading about item response theory. I was wondering if I should treat it as a three-level model with individual choice points at level one, trials at level two, and participants at level three. In this case, I would have one observation at each choice point (i.e., correct or incorrect). I’m not entirely sure what the model would look like, but I think something like this:
three.level.model <- glmer(error ~ feature + (1 | feature) + (1 | trial) + (1 | participant), data = df, family = binomial())
My main question is, Does one model better answer my research question than another model? (Maybe I should be asking, Does either model answer my research questions?)
If I understand it correctly, the two-level model treats all choice points with a given type of feature as being equivalent. The three-level model would allow me to ask additional questions (e.g., do participants make fewer errors earlier or later in the environment). Can I still reasonably answer my main question using this three-level model, though?
I appreciate any feedback or help you can provide. Thanks to so many different message boards I feel like I am gaining a better grasp of multilevel modeling, though I still have a LONG way to go!
Edit to answer some clarifying questions:
The environment is the same for all trials and all participants. Nothing changes across the trials. The environment is relatively complex (think large outdoor maze). In total, there are 15 choice points (each containing 5 of the same kind of feature). I apologize for not better explaining the errors. In the two-level model I was anticipating using the total number of errors made for the 5 choice points with a given feature. If I do the three-level model, it would be a dichotomous correct/incorrect.
