10
$\begingroup$

Many randomised controlled trial (RCT) papers report significance tests on baseline parameters just after/before randomisation to show that the groups are indeed similar. This is often part of a "baseline characteristics" table. However, significance tests measure the probability of getting the observed (or a stronger) difference by chance, aren't they? And if the test is significant we conclude that there is a true difference because a random difference of that extent would be unlikely. Does a significance test make any sense after randomisation when we know that any difference must be due to chance?

$\endgroup$
6
$\begingroup$

A hypothesis test would be nonsensical, but a significance test may be useful.

The hypothesis test would be testing a null hypothesis that is already known to be true, as your question makes clear. It is silly to apply a statistical test to any hypothesis that has a truth value already known via completely reliable information.

A significance test provides a P value that, again as you already say, indicates the probability of getting data as extreme or more extreme given the null hypothesis. However, it seems to me that such a P value can be interpreted in a manner that equates to an answer to the question "How often might I expect to see a difference in baseline values as large as this time, or larger?" The answer might be useful even if it is not clear for what purpose.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.