I am creating a term to describe a design that may not be the proper term ("Staggered within subjects design"). What I mean by this is one can imagine a within subjects design with 3 levels. Each participant provides data for two levels, and between participant the order in which the data is collected is counter-balanced. For example (condition order is in the cells of the table below; sorry for the table formatting, I did my best, but I think you'll get the gist):
Participant Condition A Condition B Condition C 1 1 2 2 2 1 3 1 2 4 2 1 5 1 2 6 2 1 7 1 2 8 2 1 9 1 2 10 2 1 11 1 2 12 2 1
Does this sort of design have a name? What are the consequences of such a design? How would one analyze it? I understand that I could probably use a mixed model to analyze this data but I'm curious about the application of ANOVA/t-test style analyses to this sort of data as these statistical approaches are more familiar to many in my field.
The rationale for such a design might be that you have limited time with each participant and so you can't run them through all three conditions however each participant must be rewarded the same whether they participate in 1 condition or 2 conditions.
It seems like, with special consideration, that this data could be run as a between subjects ANOVA (is the same as between subjects t-tests) or a within-subjects ANOVA. I will assert points, and I would appreciate it if the person to answer this question would accept or reject these points.
- I could conduct a within Subjects anova on this data comparing any two conditions crossing condition with trial order also as a within-subjects factor (I expect that the trial order effects will be non-significant).
- If there were main effects of order it would influence the validity of my estimate of the exact amount of effect of being demonstrated by each condition but would not influence subsequent analyses.
- If there were interactions between a condition and order I should not conduct any between subjects analyses on data coming from order 2.
- I could run two between subjects t-tests for each pair of conditions. One for the data from order 1 and the other from order 2 (as using data from both in a single between subjects analysis would violate the assumption of independence).
If I could run all of these statistics, using the same data I would have 3 statistical tests. Each of the between subjects tests would be a full replication of the comparison between condition A and B. The within subjects tests would validate my hypothesis that trial order does not matter and allow the between subjects tests to be applied. If I fail to find significance in my between subjects analysis my within subjects analysis may provide additional power which will allow me to evaluate whether my hypothesis is sufficiently supported to run an additional study or whether I'm barking up the wrong tree.