I'm about to analyse some data (hypothesis testing) and would like some feedback on my approach as I have never seen this situation (missing cells in an ANOVA-like table). I would also like to know if a multi-level approach would be better (I have no experience with it, just curious to learn more). The data have the following repeated measures design:
A1 A2 B1 A1B1 A2B1 B2 A1B2 . B3 A1B3 .
(Sorry about formatting). As you can see, A2B2 and A2B3 do not exist (were not tested).
The first hypothesis is that
y(A1B3) > y(A1B2) > y(A1B1), while the second is that
y(A2B1) > y(A1B1). I would also like to compare
y(A1B3), though these will be exploratory. The main reason for including this
A2B1 was to verify that something did not go wrong as there is a well established effect (hence the hypothesis
y(A2B1) > y(A1B1)). So I think that this is rather a separate question and, once verified, does not require further consideration (e.g. comparison with the other factors). That is the reason for the missing cells.
I am thinking about running a one-way ANOVA (effect of B holding A1 constant) for hypothesis 1 and a t-test (effect of A holding B1 constant) for hypothesis 2. As for the exploratory tests I was going to run two further t-tests. I think this is all ok but would like some feedback on it in case there is a problem with it.
As for the multi-level approach, the study involved a sample of participants all carrying out the same task. So it is hierarchical because there is a "single subject" level (each with ~30 trials in each condition/level), and then a "group" level on which I was planning to test the hypotheses on just the aggregated (means) single-subject data. So I wonder if multi-level is the way to go instead of ANOVA + t-tests?
Edit for clarification: there are 48 participants. By group level I just meant taking each subject's summary statistics and analysing those as in a typical ANOVA. As for the design, it is repeated measures and all subjects carry out the same task with the same conditions. Each trial is defined by belonging to one of the cells in the table.
Edit 2 for an example: Each trial of the experiment began with information about the upcoming target stimulus location, and this was either correct (A1) or incorrect (A2)*. When the target stimulus appeared in either the correct/incorrect location, it was either presented on its own (B1), with a distractor that was similar to the target (B2), or with one that was highly dissimilar (B3). The thing that makes it a bit strange is that *when a distrctor was presented, the preceding information was always correct (A1), hence the missing cells. So the factor A1/A2 was only "relevant" when there was no distractor, otherwise it was always just A1.