Disclaimer: I have a very minimal stats background and am just getting into this stuff. Corrections to any terminology I use or an improved rephrasing would be super appreciated.
I have a simple science/perception experiment that I've run, testing a single group of subjects on 2-dimensions of conditions: speed (3 possible values) and "encoding type" (3 possible values). On a given trial, a subject is presented with a stimulus of a random "encoding type" presented at a random speed. I have a small number of subjects (~10). For each subject, I've calculated their performance (an observation) for each speed+encoding type combination.
I'm interested in only seeing if there is a significant difference in performance of of "encoding type" restricted to a given speed (e.g. is encoding type 1 at speed 1 different from encoding type 2 at speed 1). I don't care of comparisons across speeds (e.g. I don't care if encoding type at speed 1 is different from encoding type 1 at speed 2). I'm doing 9 comparisons total (3 for each speed). So far, I've only run some paired ttests (would this even be the right thing to do?). I am getting some significant results without correction (p values ~0.03).
- Is a paired ttest the best test to use?
- Should I be correcting at all?
- Is there something other than bonferroni/sidak I should try? I've attempted to use bonferroni correction, where I took 0.05/9 = 0.0055. This throws out a number of results. Should I be dividing by 9 (the total number of tests I've performed) or by 3, the total number of tests constrained to the speed condition?
