Which test to use? Paired or unpaired?

Question

I have a small dataset that was gathered from 1 subject during 1 “session”. At the beginning of the session, the subject performed a movement task for approximately 40 seconds, and completed about 20 repetitions of the movement during that time.

Approximately 1 hour later, at the end of the session, the subject once again performed the movement task. This time, the subject performed the task for about 170 seconds and completed several more movement repetitions.

With each repetition of the movement we obtain a single number that represents the “force” used by the subject during that repetition.

Therefore, my small dataset consists of:

• about 20 reps from timepoint 1, each with an associated amount of force, over a 40 second period
• about 70 reps from timepoint 2, each with an associated force, over a 170 second period.

I want to see if the average force exerted by this 1 subject at timepoint 2 is significantly lower than the average force at timepoint 1.

My inclination was to use a paired t-test, but that’s impossible since a paired t-test requires matched samples.

I could “truncate” the number of samples at timepoint 2 so that it is equal to the number of samples at timepoint 1, but it’s still not truly matching the samples.

For the time being I’ve just done an unpaired t-test to compare the samples, but I’m curious what is the “correct” way to handle a sample like this.

Answer 1

This is indeed not a matched sample, so the paired t-test is wrong. I suspect though that all measurements depend on each other, given that it's the same subject performing all these tasks. The unpaired t-test assumes independence, so is arguably wrong as well. (Note that the paired t-test doesn't deal with dependence between data from different tasks either.)

The unpaired t-test would be valid assuming that the observations are independent conditionally on a constant "subject effect" (i.e., a model with a random effect that only takes one value here as there is only one subject). In that case the test would test its null hypothesis conditionally on the subject effect of the observed subject, so no generalisation to other subjects is valid. Maybe you only want to make a statement regarding the one subject that you have, in which case this could work.

However I suspect that there is more dependence going on than just a constant subject effect. There may be training or tiring effects at task level, which would invalidate the unpaired t-test as well, and a model for the dependence would be required to get anywhere.

In any case whatever you do with data from one subject it will not allow generalisation to other subjects, so if this is what you want, you can't have it based on these data.