Which formula fits better for this Linear Mixed-Effects Model?

Question

I am currently analyzing a dataset that contains a list of flight simulator tests performed by different pilots. I want to analyze if a certain flight parameter (i.e. amount of input errors during flight, lateral deviation to ideal path, etc.) are affected by the categorical variables in the following list:

1. Campaign: the pilots flew the same flights but in different places (and environmental conditions, such as lack of oxygen, isolation, etc). 5 different campaigns were done. A performance difference is likely to appear depending on the campaigns.
2. Group: in each campaign, the pilots were divided in two groups: Frequend and infrequent flyers. A performance difference is expected between both groups.
3. Session: During each campaign, the same amount of flights were performed, each month for FF pilots, or every three months for IF Pilots. In total, 10 sessions were made. A variation of performance might happen throughout the experiment, also affected by Group and Campaign.
4. Flight Scenario: three different flight scenarios were flown, which required different skill levels. The performance is also expected to vary between type of scenario. Additionally, an extra list of categorical variables could be considered (Gender, Age, Background, etc.).

Could you please tell me which LME Model formula would you better implement in order to understand the dataset presented? And if you wish, how would you better plot the results of such an analysis?

Answer 1

Building upon sjp's answer, I'd suggest a slightly more complex model that may be simplified if necessary. You've said that pilots were placed in groups so group-specific effects may occur and that perfomance differences are likely to change between campaigns which the groups were placed in. So to me it sounds like a more complex multilevel/hierarchical model that should account for the nested structure of pilots within groups within campaigns. So, I would suggest:

model <- lmer(errors ~ Campaign + Group + Session + Scenario + scale(Age) + Gender + Session:Group + Session:Campaign + (1 | Pilot / Group / Campaign), data = df)

Besides ggplot2 and effects you can also check out performance for model's performance analysis and some nice visualisations.

Answer 2

I have put the syntax below for a full(ish) model containing most of the variables you mentioned. I used the poisson, because this is used to model count data, and counting the number of errors should be modelled this way. If you have another response variable that is continuous, you can remove family = "poisson" and replace glmer with lmer.

I specify an interaction between Session and Group, and Session and Campaign, and you mentioned you expected that there could be differences. I also included a random intercept for pilot. There is a possibility that the random effects should be different, but you would need to make those decisions as a subject matter expert with knowledge of the experimental design. Here is the model:

model <- glmer(errors ~ Campaign + Group + Session + Scenario + scale(Age) +
               Gender + Session:Group + Session:Campaign + (1|pilot),
               data = df, family = "poisson")`

As for plotting the data, I would use the effects package in R, at least for exploratory plots to better understand the model. For presenting the results, I would probably use ggplot2