Aggregating and comparing data across days using paired t-test

Question

I have two groups of data (with and without stimulus) from the same subject. For each experimental session, I measure a score (let say reaction times, n_with_stim=50, n_without_stim=50). For each day, the subject participates in four sessions (hence, n_with_stim_total=200, n_without_stim_total=200). My ultimate goal is to test whether the stimulus has any effect. The problems are (i) his baseline score changes across session; The later in the day, the slower his reaction time, and (ii) his baseline score may change across day.

How can I aggregate and compare the results across sessions? Can I just combine all the data across sessions within the day to see whether there is an effect on that particular day?

Is it ok to combine data across sessions and days to compare?

Is paired t-test an appropriate test for me to use?

Thank you in advance.

Answer 1

To me it sounds like a case for simple 2-way repeated-measures ANOVA.

The results of the ANOVA will tell you how much variance is accounted for by (1)Stim condition (2) Trail number, and (3) 1x2 interaction (does stim affect the change across trials)

Tutorials for running repeated-measures ANOVA in your favorite software are pretty googlable; and/or I bet you could get software-specific advice here.

This is probably more of a 'comment' than an 'answer' but I haven't got quite enough reputation points to comment yet. :)

Answer 2

I will start with your questions in reverse order:

Paired t-test? - Possibly usable - note it relies on a normal data set. Is your data normal? Also, you will need to do a test to determine if you are getting an impact from time / date on your experiments.

Ok to combine data? - Unless you know (are confident that) outcomes don't change over time probably not. You could check this by running a repeat experiement throughout each day as a reference point.

Suggestion - consider using a statistical design for your experiment rather than trying to fit your data to a test. I have used central composite rotatable design (sounds scarry for a non-professional statistician to begin with but quite good). This would allow you to include your extra time / day variables with a mimimum number of experiments to see if they are interacting with the variables that you are most interested in. I know Minitab has this built in - other may be able to direct you to good information about how to set them up in the software you have.

Hope this helps.

Mark