I conducted an experiment in which I timed participants completing various tasks, and I'm trying to figure out what factors influenced completion time and if that influence was significant (using R).
I have followed quite a few R tutorials on ANOVA testing, but my data has a few nuances: it's a repeated measures / within subjects design (9 participants, 74 observations total); each participant did multiple tasks with and without training (my main factor of interest); and I have incomplete block design because not all participants completed all tasks. However, every task that was completed was done with & without training.
The goal is to assess whether or not training helps save time for task completion, while accounting for the different tasks and subjects as sources of variance. The data looks like this:
> df[31:40,] # A tibble: 10 × 5 TaskPerformer YearsOfExperience TaskID TrainedForTask TotalSeconds <fct> <dbl> <fct> <fct> <dbl> 1 Tester5 15 4 TRUE 230. 2 Tester9 16 4 TRUE 116. 3 Tester3 20 5 TRUE 305. 4 Tester3 20 1 TRUE 302. 5 Tester2 12 1 TRUE 293. 6 Tester9 16 1 TRUE 272. 7 Tester3 20 3 TRUE 251. 8 Tester7 15 2 FALSE 190. 9 Tester7 15 4 FALSE 159. 10 Tester3 20 1 FALSE 149.
The block design:
> xtabs(~ TaskPerformer + TaskID + PreppedForTask, data = df) , , PreppedForTask = FALSE TaskID TaskPerformer 1 2 3 4 5 Tester1 0 1 1 1 1 Tester2 1 1 1 1 1 Tester3 1 1 1 1 1 Tester4 0 1 1 1 0 Tester5 0 1 1 1 1 Tester6 0 1 1 1 1 Tester7 0 1 1 1 0 Tester8 1 0 1 1 1 Tester9 1 1 1 1 1
"PreppedForTask = TRUE" is similar.
An incomplete list of what I've tried:
afex::aov_car(TotalSeconds ~ TrainedForTask*TaskPerformer + Error(TaskPerformer), data=df). This doesn't work because the testers do multiple tasks. Error msg:
"Following ids are in more than one between subjects condition: Tester1, ... Tester9". Also, it doesn't take into account the TaskID. Some tasks are more difficult than others and could impact time taken.
- 3-way ANOVA with the TaskID and TrainedForTask as 2 'within-subjects' factors + TaskPerformer as the 1 'between-subjects' factor. First of all, I'm not even sure that's the right set up, but it doesn't work because of the incomplete block design. I could do a 1- or 2-way ANOVA, but if I just look at TrainedForTask and TotalSeconds, the training doesn't have a significant impact - I need to know how TaskPerformer and TaskID affect the results.
- Based on this question, I tried:
anova(lmer(TotalSeconds ~ TrainedForTask + TaskID + (1|TaskPerformer), data=df))which ran without errors but shows no statistical significance, which is surprising because a paired, one-sided t-test (assuming training improves task time) does have p-val < 0.05. And similar to the person who asked that question, I am not sure I should be treating TaskPerformer like this or if it should be included as a main effect.
I'm happy to share more detail if needed. I have seen lots of similar questions but nothing that has this combo of multi-factor, incomplete block design, and repeated measures with multiple tasks per subject. Thanks in advance for your thoughts.