Paired vs unpaired samples t-test in crossover study

Question

I have to analyse a 2x2 AB/BA crossover study, in which every participant was administered drug A and then no drug at all, or vice versa. There were two study visits and in each visit the peak glucose values after an oral mixed meal test were taken for each participant.

I am confused about the type of t-test I should use to find the treatment effect. Some resources suggest an unpaired samples t-test link here,whereas the majority of previous studies I've read uses paired samples t-tests.

I understand that comparing all observations from Period 1 to Period 2 would require a paired samples t-test, but would that calculate the treatment effect or the period effect?

Answer 1

An unpaired t-test uses the difference between the means of the control and test datasets to determine the p-value. A paired t-test combines the control and test data first, by taking the difference between control and test values for each individual experimental unit, and then compares the mean of those differences to the theoretically expected value (i.e. null hypothesis value). In other words, the paired test constructs a dataset of differences and then does a one sample test.

For your circumstance the experimental units are the subjects and the control and test conditions are drug and no drug.

The advantage of the paired test comes into play when there is variation that is shared across the control and test situation, and thus can be removed in the within subject differencing.

The reduction in variation by pairing gives more power to the paired test, but at the cost of sample size. Twenty observations from ten subjects measured in two conditions gives 9 degrees of freedom for the paired t-test, whereas 20 measurements analysed in an independent arrangement gives 18 degrees of freedom. That means that if there is no shared variation to be removed by the pairing, the unpaired test has a little more power. However, you do not need to reduce the variation m uch to overcome that small power advantage.

In most cases if there is a 'natural' pairing between the control and test measurements then using the paired test is the best approach. Your situation is nearly as straightforward as a before and after comparison within subject and so the paired test is going to be sensible.

Answer 2

The Deutsches Ärzteblatt article recommends a two-stage analysis of cross-over trials:

1. Run a preliminary test for carry-over effect between periods 1 and 2. Let's call this test SEQ.
2. Conditional on the result:
   a) If the carry-over is not statistically significant, run a within-patient test for treatment effect on the differences A - B between period 1 and period 2. Call this test CROSS.
   b) Otherwise, run a between-patient test for treatment effect on the difference between groups A and B in period 1 (& discard the period 2 observations). Call this test PAR.

CROSS corresponds to a paired t-test (more sophisticated analyses are possible; see "Cross-over Trials in Clinical Research") while PAR corresponds to an unpaired t-test.

In "Cross-over Trials in Clinical Research" Stephen Senn explains why

This procedure (is) potentially misleading and unsatisfactory. The two-stage procedure is therefore of historical rather than scientific interest and can no longer be regarded as a serious option for analysis.

The reasons include:

• Failure to reject the null hypothesis of no carry-over doesn't prove there is no carry-over. So the preliminary test is pointless. Instead, plan for sufficient washout and assume no carry-over effect.
• The carry-over test SEQ and the between-patient test PAR are highly correlated: If the carry-over effect is "significant", there is high probability the treatment effect is significant also. Consequently, the two-stage procedure's type I error is higher than the nominal $\alpha$ level.
• The PAR test is less efficient than the CROSS test.

References

• Wellek S, Blettner M. On the proper use of the crossover design in clinical trials: part 18 of a series on evaluation of scientific publications. Deutsches Arzteblatt International. 2012 Apr;109(15):276-281. DOI: 10.3238/arztebl.2012.0276.
• Senn S. Cross-over Trials in Clinical Research. Statistics in Practice. Wiley, 2002.
• Senn S. The AB/BA Cross-over: How to perform the two-stage analysis if you can't be persuaded that you shouldn't. Published in: Hansen, B and De Ridder, M. Liber Amicorum Roel van Strik, Erasmus University, Rotterdam, 1996, pp93-100. Available at http://senns.uk/Blogs.html.

Answer 3

When dealing with before/after for the sample participant, it is a paired test, when comparing different participants then it is an unpaired test.
It looks like you have both a paired test, in the AB and BA parts, and then an unpaired test, comparing AB to BA part