2
$\begingroup$

I want to analyze the difference between two paired samples. My initial plan was to compare them using the Wilcoxon test because I didn't want to bother checking whether the data is properly distributed for (e.g) paired t-test, and selecting a non-parametric test sounds like a safer mode of action. What is the drawback of my laziness? What am I risking?

$\endgroup$
4
  • $\begingroup$ Lower power... less likely to detect an effect, if an effect is present. $\endgroup$ Apr 22, 2021 at 7:09
  • $\begingroup$ @user2974951 Not necessarily. If the parametric test is not appropriate the power could be lower for that test. Boris, if you have enough data, e.g., 40 entries, plot a histogram and look at it, also, tests for which distribution it is are very easy to perform, if there is enough data. Also, if you don't have enough data and if you know something about the underlying physics that give rise to the data, you may already know what type of distribution it has to be. $\endgroup$
    – Carl
    Apr 22, 2021 at 7:19
  • $\begingroup$ @StephanKolassa Yes, thank you! $\endgroup$ Apr 22, 2021 at 8:05
  • $\begingroup$ The link is informative and comprehensive. You should read it, but your question is just about paired tests and my answer focuses on them. $\endgroup$
    – BruceET
    Apr 22, 2021 at 8:23

1 Answer 1

2
$\begingroup$

The first difficulty with your question is the implication that it is always appropriate to use the nonparametric test without checking anything, while it takes considerable effort to discover whether it is OK to use a t test. It is true that a nonparametric test does not require normal data, but nonparametric tests have their own requirements. For example, a Wilcoxon signed rank test works best when data are symmetrical, and does not work well if there are ties--especially in small samples.

Whether you use a Wilcoxon signed rank test or a t test, a paired test is essentially a one sample test on differences, so it is easy to compare the power of the two tests in case data are normal. It is a good idea to look at a stripchart, boxplot, and normal probability plot of the data before deciding which test to use.

The issue of power has been discussed briefly in comments. If data are normal and not rounded to give ties, both tests may OK, but the t test will have the better power.

Suppose you have a sample of size $n = 12$ pairs with differences distributed $\mathsf{Norm}(\mu=2,\sigma= 2)$ then what is the probability I will detect that the mean is not $0,$ using a t test with significance level $\alpha = 0.05 = 5\%?$ The answer by simulation is about $0.88 = 88\%.$

set.seed(422)
pv = replicate(10^5, t.test(rnorm(12,2,2))$p.value)
mean(pv <= .05)
[1] 0.88211

If the normal data are rounded to integers, then the t test gives almost the same power, about $87.5\%.$

set.seed(422)
pv = replicate(10^5, t.test(round(rnorm(12,2,2)))$p.value)
mean(pv <= .05)
[1] 0.8745

If I use a Wilcoxon signed-rank test on the same data, what is the probability I will detect that the data are not centered at $0?$ With the nonparametric Wilcoxon test the power is only about $0.78 = 78\%.$

set.seed(422)
pv = replicate(10^5, wilcox.test(rnorm(10,2,2))$p.value)
mean(pv <= .05)
[1] 0.78466

In this case, if the data are rounded to integers, than there will be warnings about the ties induced by rounding, P-values may not be accurate and the power will be below $75\%.$

set.seed(422)
pv = replicate(10^5, wilcox.test(round(rnorm(10,2,2)))$p.value)
mean(pv <= .05)
[1] 0.74398

On average, there are more than four ties out of 12 differences distributed $\mathsf{Norm}(2,2).$

set.seed(422)
nr.tie = replicate(10^5, 10-lengh(unique(round(rnorm(10,2,2)))) )
mean(nr.tie)
[1] 4.23821
$\endgroup$
1
  • 1
    $\begingroup$ Thank you very much. Very helpful $\endgroup$ Apr 22, 2021 at 10:03

Not the answer you're looking for? Browse other questions tagged or ask your own question.