I'm running an experiment with an e-learning system using a repeated measures with crossover design.
Subjects are shown two videos on a web page. In one session, a feature (a table of contents) is enabled, while the other session is a control:
C=Control F=Feature subject pretest1 video 1 posttest1 pretest2 video 2 posttest2 S1 C F S2 C F S3 C F S4 F C S5 F C S6 F C
The amount learned is measured by subtracting pre-test result from post-test result.
My issue is that I can't swap around the order of the two videos for the crossover because Video 2 requires knowledge of Video 1. If I had chosen videos on separate topics I would be introducing variability in aptitude, interest and prior knowledge within the subjects.
It is also unlikely that both videos have equal difficulty, so the learning gains in one can't be directly compared to the learning gains in the other. I know that the crossover will cancel out this effect and allows me to get a mean difference, but this seems to prevent me from doing basic EDA like scatterplots, and I'm not sure how to proceed with statistical analysis from there.
Is there anything that I can do to improve this experiment design, and do you have any advice or resources on how I should proceed with analysis in this scenario?