6
$\begingroup$

The title is my question, here is a little more information. I was checking off on an analysis done by someone else and they used a paired t-test on data from the same subject; however the difference that was calculated for the paired t-test was from two different variables (the variables are not measurement 1 and 2 of the same thing, they represent measurements of the same idea, energy expenditure measured in two different ways. Variable A is expenditure measured using method A and variable B is expenditure measures using method B).

Each subject had a measurement taken on them using method A and a measurement using a different method method B (these methods theoretically measure they same thing, but one is the gold standard and the other is not) and then the difference was calculated as method B - method A and the paired t-test was done on the vector of the differences.

My question for this specific data set is, is this a valid pair? Typically, I see paired t-tests using the same variables, for instance pre-post measures on an individual variable for each experimental unit, or twin studies where the same variables are collected.

More generally my question is what constitutes a valid pair for a paired t-test? Is it valid to consider two different variables (for instance variable for method A and a variable for method B) a pair? And if so what if any issues can arise when a paired test is being performed on different variables.

Improve this question
$\endgroup$
7
  • $\begingroup$ You may wish to read up on individual matching, and on both the randomized cross-over study design, and the case-cross-over study design, as these will bear on both repeated measures within individuals, and on matching-based pairing. Good question. $\endgroup$
    – Alexis
    Commented Feb 17, 2020 at 17:08
  • $\begingroup$ One wonders what exactly you mean by "two different variables," given that it's routine to use a distinct random variable to model every observation in an experiment. Could you perhaps be referring to variables whose values are expressed in different units of measurement, or do you mean something else? $\endgroup$
    – whuber ♦
    Commented Feb 17, 2020 at 17:20
  • $\begingroup$ @Alexis thanks I will look more into all those topics. $\endgroup$
    – RAND
    Commented Feb 17, 2020 at 17:21
  • 2
    $\begingroup$ I remember being amused to learn, long ago, that it can make sense to apply a paired t-test even when the variables seem to have no relationship at all. Provided they are positively correlated, the paired t-test will be more powerful than the unpaired one (subject, as usual, to the assumptions needed for a t-test to be appropriate in the first place). Indeed, when you read about the assumptions and derivation of the t-test, you won't see any restrictions whatever on the meanings of the values being compared. $\endgroup$
    – whuber ♦
    Commented Feb 17, 2020 at 17:30
  • 1
    $\begingroup$ Well, when you don't expect variables to be associated with one another, then it can backfire to use a paired test. Bear in mind that by "positively correlated" we are referring to the underlying bivariate distribution rather than what you might see in the data. However, whenever you are measuring multiple properties of individuals in an experiment, there often are good scientific reasons to suppose those measurements will be correlated (and those reasons should suggest the sign of the correlation, too). $\endgroup$
    – whuber ♦
    Commented Feb 17, 2020 at 17:52

1 Answer 1

Reset to default
1
$\begingroup$

Yes, the two measurement methods could constitute a "pair" in this example. More concerning (in my opinion) is that a t-test will only help you identify a constant bias between methods A and B. There could be a proportional (and/or constant) bias(es) between the methods which a t-test is not well suited to identify. Consider the data

$A = (10, 11, 12, 13, 14)$

$B = (11, 11.5, 12, 12.5, 13)$

A paired t-test on these data will not be significant:

$diff = 0[-0.982, 0.982]$

but both the proportional and constant biases are being missed.

CLSI EP09 describes various ways to perform method comparisons.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.