1
$\begingroup$

I have been asked by a reviewer to perform a paired sample t-test, rather than an independent sample t-test, for two groups (males vs females) who are matched for a physiological characteristic (fitness, essentially).

In these groups, participants were paired based on fitness levels (e.g. participant 1 for males was paired with participant 10 for females based on their similar relative fitness levels). So, the reviewers suggestion does make sense to me, but I'm not sure it's the correct approach.

Any help is much is much appreciated.

$\endgroup$
1
  • 1
    $\begingroup$ What is the response or outcome variable? Fitness or something else? $\endgroup$ – AdamO Apr 22 at 17:34
1
$\begingroup$

Cautionary tale: You have to be very careful to guard against 'false discovery' when contriving 'fake pairs' based on a criterion introduced after data are available.

Consider the fictitious data below (sampled in R) with $n=20$ scores in y1 and `y2' displayed below:

y1
 [1]  98 108 105 105 113  71 104 114 100 126
[11]  84  96 103 123 124  72 124 102 122 123
y2
 [1]  72  74  87 100 103 106 109 111 113 115
[11] 116 120 126 128 137 139 140 142 143 146

boxplot(x1,x2, horizontal=T, col="skyblue2")

enter image description here

It seems fair to assume that the two samples are uncorrelated and nearly normally distributed. Normal probability plots and a scatterplot are shown below and the correlation is about $r = 0.22.$

cor(y1,y2)
[1] 0.2249666

enter image description here

A Welch 2-sample t test finds no significant difference in means at the 5% level.

t.test(y1,y2)$p.val
[1] 0.09639152

A paired t test on the data arranged in the order recorded, shows no significant difference at the 5% level.

t.test(y1,y2, pair=T)$p.val
[1] 0.0688505

However, if I decide after seeing the data, to pair the data in sort order, then a paired t test show a bogus 'very highly significant' difference with P-value very near $0:$

t.test(sort(y1), sort(y2), pair=T)

        Paired t-test

data:  sort(y1) and sort(y2)
t = -7.9373, df = 19, p-value = 1.884e-07
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -13.268811  -7.731189
sample estimates:
mean of the differences 
                  -10.5 
$\endgroup$
4
  • $\begingroup$ Can you say more about the rank-paired significant T-test? This is really just a matter of a skewed data, correct? I suspect symmetric samples will actually provide a test of the correct level when using a rank paired T-test. $\endgroup$ – AdamO Apr 22 at 19:22
  • $\begingroup$ Mostly it is a matter of 'creating' significance in a paired test when "pairing" is done just by looking at the data. I don't believe skewness entered into my bogus pairing. $\endgroup$ – BruceET Apr 22 at 19:26
  • $\begingroup$ This is interesting and got a +1 from me, but I read the original question to mean that the pairing is based on some other factor, not something in the data you gave. $\endgroup$ – Dave Apr 22 at 19:57
  • $\begingroup$ @Dave. Thanks for comment. I'm still a bit skeptical. The referee who's making the recommendation to pair has seen the data. If the pairing was done with care, and the referee understands and believes that, then it's probably OK. Ideally the original expt protocol would say explicitly how pairing is done and envisioned a paired test. // Was pairing done meticulously so that we can view 'pairs' as truly meaningful, or was pairing done in some rough sense to try for 'similar' groups (not trusting randomization to do its job). In the latter case we have neither true paring nor true randomization. $\endgroup$ – BruceET Apr 22 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.