I know it sounds abysmally ignorant, but here we go.

I understand the basic logic behind the t-test: you want to know whether the difference in the mean of two samples is due to chance or not. To this end, you need to take variability into account. But variability at which level?

Suppose I want to run a dependent t-test on an experiment I've made. I have 20 athletes who each run 10 races without EPO, then 10 races with EPO in their blood. My question is whether EPO makes any difference in their performance.

I have two levels of variability in my data: intra-subject variability (the difference in performance between two races run by the same athlete) and inter-subject variability (the variability in the athlete's average performances).

What shall I do? Compute the average performance of each athlete for each condition, and then compute the t-test from this, or work directly from the raw data?

  • 2
    $\begingroup$ If you have not yet heard of Analysis of Variance (ANOVA), then you will benefit by conducting a quick review of it. $\endgroup$ – whuber Sep 24 '14 at 17:24

I don't think this is so abysmal. It takes some sophistication to recognize this discrepancy if you are primarily familiar with the $t$-test. There are actually two levels of dependency in your situation: the same runners are assessed in both of two conditions, and there are multiple (viz, 10) measures of each runner in each condition. Let's start with some simpler studies than the one you conducted and work our way up:

  1. Randomly assign each runner to either the EPO group or the no EPO group and have them run the race once.
    This is a two, or independent, samples $t$-test
  2. Have each runner run the race twice (hopefully with enough time in between to recover fully), but once with EPO and once without.
    This is a paired, or dependent, samples $t$-test
  3. Randomly assign each runner to either the EPO group or the no EPO group and have them run the race ten times.
    This is an ANOVA with one between subjects factor (EPO vs. not) and one within subjects factor (race).
  4. (Your design) Have each runner run the race 20 times, 10 with EPO and 10 without.
    This is an ANOVA with two within subjects factors (EPO and race).
  • $\begingroup$ In this case, we have two levels in the factor (EPO vs not). Isn't this equivalent to running a t-test rather than ANOVA for difference between the means? $\endgroup$ – SmallChess Sep 25 '14 at 5:02
  • $\begingroup$ @StudentT, you are doing 2 things, so you need a model that does that. You need to run a repeated measures ANOVA (or higher / more complicated version). W/i that model, this is analogous to running a (dependent) t-test; I wouldn't quite say it is "equivalent", but that may be just semantics. $\endgroup$ – gung - Reinstate Monica Sep 25 '14 at 14:41
  • $\begingroup$ Thanks. Do I still need to run an ANOVA with two factors even if I am not interested in the effect of the specific races? The point of having the participants run several races is to increase the number of data points and hence the reliability of measurement, not to investigate whether the second race is easier than the third. $\endgroup$ – Sophonax Sep 30 '14 at 15:29
  • $\begingroup$ @Sophonax, yes, the model needs to account for the fact that the races within a runner are not independent. If you didn't run the two-way repeated measures ANOVA, you would be treating those data as independent, which is invalid. In this case, race is just a nuisance variable; that's not uncommon in data analysis. $\endgroup$ – gung - Reinstate Monica Sep 30 '14 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.