Guide me on evaluating the data collected: t-tests too basic? Something more robust?

Question

Two similar groups read short passages and had their accuracy scores and times recorded.

Prior to reading the passages each group (A and B) underwent a treatment. Since the N size was low, I decided not to use a control vs. experimental design... instead, I tried to do a within groups study. So, for the study described here, one group read passages 1 to 20 in (ascending order) and the other group read passages 20 to 1 (in descending order). To try to keep the experiment clean, I reversed the treatment for each group alternating treatments as they cycled thru the readings. For example, when Group A did Reading 1 they had treatment X, but when Group B did Reading 1 they had treatment Y. When Group A did reading 2 they had treatment Y, and when Group B did reading 2 they had treatment X. As noted, rather than having each group start at 1 and go to 20, I reversed the order to avoid any potential "practice effect" occurring.

What I am hoping to find out by analyzing this data is whether or not the treatments had any effect on the speed and accuracy of the participants.

My knowledge of statistics is extremely limited, so much so that I don't really know how to ask the question coherently. I am hoping someone can point me in the right directions with some helpful words, phrases, or directions so that I may read up and teach myself what to do to analyze my data sets. Basically I would like to know how to analyze the data I have gathered. I am pretty sure this is not suitable for t-tests... so what does that leave me? I'm assuming it's something a little more robust like spss or R?

Answer 1

First, SPSS and R are not "something more robust". Each of those are programs that can do many kinds of statistics, including t-tests.

Second, if you are, as you say, very new to statistics, then I would start by taking a couple courses and reading some books. One of my favorite books for intro stats is Statistics by Freedman, Pisani, and Purves. It doesn't dumb things down, but won't overwhelm you with formulas. But it will make you think.

Third, for this particular analysis, I think it would be better for you to hire a consultant (not me, I'm retired) or, if you are at a university, maybe find someone in the statistics department who will help for a co-authorship (if you are planning to submit this somewhere). I'm all for learning by doing, but I think that people need guidance in what to do first and second and so on.

Finally, although I am not certain, it sounds like you may have created an example of a famous quote from Ronald Fisher:

“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.”

From your description, both groups had both treatments, they just varied in when they had each treatment (one group had X before 1, 3, etc. and Y before 2, 4, 6 etc. and the other group the reverse). Then, unless the treatments have an extremely short effect, both groups had both treatments, 10 times each. It's going to be very hard to find something there, especially with a small N.

Sorry.