# Wilcoxon and Paired t test results conflict

I have mean force data from patients gameplays, and I have to find if there is significant change in force from initial to final gameplays. I have applied paired t test and wilcoxon tests on the initial and final gameplays, but I am getting different answers from both the test. I have also performed shapiro-wilk test and levene's test to know the variability and homogeneity of the data for parametric test (paired t test). But I am getting different results from parametric and non parametric results.

Results from paired t test:

Results from wilcoxon signed test:

Initial and final values for patient 08 and patient 09 are given below:

Patient 08 initial values: [623.98206201 850.44482101 571.66360585]

Patient 08 final values: [388.42733494 531.10230857 214.42573011]

Patient 09 initial values: [126.80567689 243.79344067 245.65232655]

Patient 09 final value: [378.69748506 386.44234597 527.17909838]


The t-test and the Wilcoxon are obviously calculated differently.

Both tests are considered tests of "location", where location is some abstract generalization of the "center" of a distribution. Actually, the Wilcoxon is a test of the median if the distribution is symmetric, in which case, if the mean exists, the mean and the median are the same thing. So they both have the same null hypothesis if the true median equals the mean. Ironically, the Wilcoxon is often advocated for (erroneously at that) in the case of data that are skewed, or, more generally, to robustify the inference for the location. Therefore, it's bizarre to run both tests when there's any confusion about the distributional assumptions that might lead to selecting one test over the other. In fact, the T-test is a distributionally robust test, perfectly suited to infer a difference in means even in the case that the data are skewed, the only requirement is a suitable sample size to ensure convergence. One may always consider logarithmically transforming the response to reduce the influence of skewness.

Among the many reasons that "running multiple tests is confusing" you now have three pinch points, the inference of test 1, the inference of test 2, and the discordance of their inferential results. The least statistically valid approach in this case is to select the test that provides the best inference. Unfortunately, now that you have seen the results, it is very hard for you to remain impartial as a scientist. On the other hand, if you had prespecified which test was a primary inference and which was a sensitivity analysis, we could have a more informed discussion on the discrepancy. For instance, if the Wilcox is non-significant and is the primary inference, and t-test is a sensitivity analysis, you would explain

our primary inference was not statistically significant. The T-test sensitivity analysis was statistically significant. No correction for multiple testing was applied. More data collection and analysis is needed. We found:

Then one of:

• The t-test was highly influenced by outliers
• The sample size was very small

The concordance of $$p<0.05$$ is a bad way to assess the agreement of two tests or two experiments. If these tests agreed in inference all the time, there would be no point in both tests existing. Running both tests "just to check" is confusing for reasons listed above. It's safest and easiest to consider that these tests fundamentally test different things and leave it at that.

Welcome to CV.

A few issues. First, Wilcoxon and t ask different questions, so, it's not surprising that they get different answers. t tests are about the mean, Wilcoxon tests pseudomedians (edited per comment).

Second, tests of normality with N = 3 are pretty much useless.

Third, more generally, when N = 3, it's going to be hard to find statistical significance. In particular, if you look at how Wilcoxon is calculated, it's going to be almost impossible.

$$T = \sum (sgn(X_i) R_i$$

where $$X_i$$ are the values and $$R_i$$ are the ranks. This will be an integer between -3 and 3. I couldn't find a table of the distribution under the null that included N = 3 (smallest was N = 5).

Finally, the difference is also explained by the nature of the tests. The means are quite different, and that matters for t-test, but not for Wilcoxon. E.g. we could change the values:

x <- c(10000, 11000, 12000)
y <- c(388.42733494, 531.10230857, 214.42573011)

wilcox.test(x, y, paired = TRUE )


and we still get p = 0.25.

You really need more data.

• I wouldn't say that Wilcoxon signed-rank tests medians. It's more related to the pseudomedian but better to say it tests whether values from one group tend to be larger than values from the other group (for the unpaired case) or for the paired case tests whether the probability is 0.5 that the mean of two randomly chosen differences exceeds zero. Better to use the rank difference test though. The pseudomedian is the median of all possible pairwise averages. Commented Aug 29, 2023 at 12:17
• Thanks @Peter for the answer, this means I have atleast 5 samples for the wilcoxon signed test?
– user395461
Commented Aug 29, 2023 at 12:19
• I was using 25 percent initial and 25 percent final values to perform significant test. I have change the percent to 45, meaning take 45 percent data as initial gameplay and last 45 percent data for final gameplays.
– user395461
Commented Aug 29, 2023 at 12:27
• Now my initial and final values are. p8 initial=[623.98206201 850.44482101 571.66360585 952.23395805 844.61367276] p8 final=[590.58977485 478.70782173 388.42733494 531.10230857 214.42573011] p9 initial= [126.80567689 243.79344067 245.65232655 309.0489331 295.31614513] p9 final=[350.38457992 415.88156994 378.69748506 386.44234597 527.17909838] now there are 5 sample data, but results are still same. Patient Median Initial Value Median Final Value Significant Difference Force statistic p Cohen's_d P08 844.61 478.71 No Decreased 0.0 0.06 NA P09 245.65 386.44 No Increased 0.0 0.06 NA
– user395461
Commented Aug 29, 2023 at 12:29
• Here is an example for the difference between the t- and the Wilcoxon test (which indeed does not test medians - depending on your definition of medians, my example can easily be modified to show that). Commented Aug 29, 2023 at 12:51