3
$\begingroup$

Suppose I have paired data. The differences are however not normally distributed (i.e., we cannot use paired t-test), but the individual data for the two groups are normally distributed and they seem to have equal variances (result from doing a test). Is it better to use unpaired t-test then (since we seem to meet all the assumptions), or is Wilcoxon signed-rank test the better option? I am not sure if I may use paired data for the unpaired t-test.

$\endgroup$
2
  • 1
    $\begingroup$ Since you seem to be very flexible in what kind of hypothesis to test, you will be most probably better off with a test that uses paired information (higher power). A Wilcoxon signed-rank test is an option, or a permutation t-test. $\endgroup$
    – Michael M
    Commented Dec 9, 2023 at 10:36
  • 1
    $\begingroup$ "since we seem to meet all the assumptions" -- except independence, right? $\endgroup$
    – Glen_b
    Commented Dec 9, 2023 at 16:16

2 Answers 2

13
$\begingroup$

No data are ever normally distributed. Statistical model assumptions are idealisations, and if model assumptions really had to be true, we couldn't do any statistical analysis ever. So it's not true that you "cannot" use the paired t-test. For large samples the t-test can be justified referring to the Central Limit Theorem, and in general there are specific deviations from normality that cause problems, such as outliers or strong skewness, whereas other deviations from normality are harmless. Without seeing the data we cannot tell whether the paired t-test is problematic here or not.

If the data are really paired, using an unpaired t-test is quite certainly worse than using a paired one; the assumption of independence for an unpaired test is usually a bigger problem than the normality assumption. As Michael M. wrote in the comments, The Wilcoxon signed rank test is an option as well as a permutation t-test, but note that the latter (as the standard paired t-test) is based on statistics that are sensitive to outliers, so if there are strong outliers I'd prefer Wilcoxon.

$\endgroup$
4
  • 2
    $\begingroup$ Since the CLT is a limit theorem that applies to type I assertion probability $\alpha$ but not to type II assertion probability $\beta$ (one minus the power), it is not usually useful in real data settings. For example the CLT can’t fix a problem where you should have taken logs but didn’t. But the assumption of independence is key as you said. I recommend a better replacement for the Wilcoxon signed-rank: the rank difference test exemplified here. $\endgroup$ Commented Dec 9, 2023 at 12:34
  • 1
    $\begingroup$ @FrankHarrell If you specify a fixed alternative with finite second moments and i.i.d. the CLT will apply indeed. There are situations (for example with outliers) where inference about the mean is misleading and the CLT argument won't work to justify the test, but from a theoretical point of view this is as much a problem with the null as with the alternative. (The only difference is that under the null with outliers chances are we'll have overconservativity, which may not be seen as a problem in itself.) $\endgroup$ Commented Dec 9, 2023 at 12:56
  • 1
    $\begingroup$ I hate to bring up an old argument but "apply" doesn't mean "it works". The accuracy of the CLT may be completely inadequate, and power is more an issue because if you should not be using a standard deviation for example, because of skewness, $\alpha$ may still be OK using the SD but $\beta$ may be destroyed. $\endgroup$ Commented Dec 9, 2023 at 16:20
  • 2
    $\begingroup$ @FrankHarrell There is probably not that much of a disagreement. There are non-normal distributions for which the t-test will work well even for small $n$, some where it won't do well for small $n$ but better for large $n$, and some for which $n$ may even be large and things don't work out. The role of the CLT in this is maybe not very central, but where things work one can at least cite it to give some theoretical justification to what is done. $\endgroup$ Commented Dec 9, 2023 at 16:57
9
$\begingroup$

You do not meet "all the assumptions" of an unpaired t-test. One assumption is that the data are independent and yours are paired.

In addition, an unpaired t-test on paired data is almost always going to have much lower power than an appropriate test. Let's use the example from R help:

sleep2 <- reshape(sleep, direction = "wide", 
                  idvar = "ID", timevar = "group")
t.test(Pair(extra.1, extra.2) ~ 1, data = sleep2)

which yields:

t = -4.0621, df = 9, p-value = 0.002833

if we use unpaired instead,

t.test(extra~group, data = sleep)

We get

t = -1.8608, df = 17.776, p-value = 0.07939

But if we follow the recommendations to use Wilcoxon:

wilcox.test(Pair(extra.1, extra.2) ~ 1, data = sleep2)

we get the much more reasonable result:

V = 0, p-value = 0.009091

albeit with warnings about ties making it impossible to get exact p values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.