Firstly I thank anyone willing to read this.
I conducted an experiment using a 2x3 repeated-measures experiment. The idea was to see if reported workload is influenced by mental states. I defined three mental states (focused on the decision to make, thinking about something related to the task and mind wandering) that I will name Focus, Around and Mind wandering. I also have two conditions, named Risky and Safe. I asked every 2 minutes on average "where is your attention?" and "what is your perceived workload?". Subjects could answer using a 5-points Likert scale from "low workload" to "high workload".
This image allowed me to see that apparently there was a significant difference in workload between conditions, but only when subjects reported being Focus or Around, not in Mind wandering. I would like to test that further, and here is the problem.
Since the data for each mental state is of different length (subjects were more or less prone to experience one mental state), I cannot use a normal paired t.test. I use nested mixed-effect models to look at it. However a colleague told me about bootstrap and that it could solve my problem. I searched a method on the internet and here is what I found.
Say that I want to compare conditions only using data relevant to mind wandering. To compare two means using bootstrap, I can create a function that will randomly assign a condition to each mind wandering related report. Then I compute the mean for the two artificial groups. I do that 10.000 times and I find the probability density function. Thanks to the central limit theorem, my density function is normal (or close). That allows me to look at the 95% confidence interval of my density function: if the real difference is included, then I cannot conclude for any effect. However if my difference is not included in the CI, then I can say that the difference I obtained is highly unlikely only by chance, therefore there must be an effect of the condition over workload.
Spoiler: I conducted this procedure using the
boot package. I use
boot.ci to get the 95% CI, however it gets me 4 different CI without any explanation (basic, normal, percentile, BCa). I found the definition of the percentile bootstrap on the internet, however no one mentions how are the other calculated. My real difference of workload is: 0.07. My output is:
So 1) Is this a correct way to use bootstraps? If so, how can I report it for publication?
2) Should I conduct a t-test at the end instead of looking at the CI?
3) In general, in what cases is this type of bootstrap better or worse than running a linear modeling? I.e should I do it here or stick to the mixed-effect model?
Thanks anyone for the help, have a great day