Wilcoxon and Paired t test results conflict

Question

I have mean force data from patients gameplays, and I have to find if there is significant change in force from initial to final gameplays. I have applied paired t test and wilcoxon tests on the initial and final gameplays, but I am getting different answers from both the test. I have also performed shapiro-wilk test and levene's test to know the variability and homogeneity of the data for parametric test (paired t test). But I am getting different results from parametric and non parametric results.

Results from paired t test:

Results from wilcoxon signed test:

Initial and final values for patient 08 and patient 09 are given below:

Patient 08 initial values: [623.98206201 850.44482101 571.66360585]

Patient 08 final values: [388.42733494 531.10230857 214.42573011]

Patient 09 initial values: [126.80567689 243.79344067 245.65232655]

Patient 09 final value: [378.69748506 386.44234597 527.17909838]

Answer 1

The t-test and the Wilcoxon are obviously calculated differently.

Both tests are considered tests of "location", where location is some abstract generalization of the "center" of a distribution. Actually, the Wilcoxon is a test of the median if the distribution is symmetric, in which case, if the mean exists, the mean and the median are the same thing. So they both have the same null hypothesis if the true median equals the mean. Ironically, the Wilcoxon is often advocated for (erroneously at that) in the case of data that are skewed, or, more generally, to robustify the inference for the location. Therefore, it's bizarre to run both tests when there's any confusion about the distributional assumptions that might lead to selecting one test over the other. In fact, the T-test is a distributionally robust test, perfectly suited to infer a difference in means even in the case that the data are skewed, the only requirement is a suitable sample size to ensure convergence. One may always consider logarithmically transforming the response to reduce the influence of skewness.

Among the many reasons that "running multiple tests is confusing" you now have three pinch points, the inference of test 1, the inference of test 2, and the discordance of their inferential results. The least statistically valid approach in this case is to select the test that provides the best inference. Unfortunately, now that you have seen the results, it is very hard for you to remain impartial as a scientist. On the other hand, if you had prespecified which test was a primary inference and which was a sensitivity analysis, we could have a more informed discussion on the discrepancy. For instance, if the Wilcox is non-significant and is the primary inference, and t-test is a sensitivity analysis, you would explain

our primary inference was not statistically significant. The T-test sensitivity analysis was statistically significant. No correction for multiple testing was applied. More data collection and analysis is needed. We found:

Then one of:

• The t-test was highly influenced by outliers
• The sample size was very small

The concordance of $p<0.05$ is a bad way to assess the agreement of two tests or two experiments. If these tests agreed in inference all the time, there would be no point in both tests existing. Running both tests "just to check" is confusing for reasons listed above. It's safest and easiest to consider that these tests fundamentally test different things and leave it at that.

Answer 2

Welcome to CV.

A few issues. First, Wilcoxon and t ask different questions, so, it's not surprising that they get different answers. t tests are about the mean, Wilcoxon tests pseudomedians (edited per comment).

Second, tests of normality with N = 3 are pretty much useless.

Third, more generally, when N = 3, it's going to be hard to find statistical significance. In particular, if you look at how Wilcoxon is calculated, it's going to be almost impossible.

$T = \sum (sgn(X_i) R_i$

where $X_i$ are the values and $R_i$ are the ranks. This will be an integer between -3 and 3. I couldn't find a table of the distribution under the null that included N = 3 (smallest was N = 5).

Finally, the difference is also explained by the nature of the tests. The means are quite different, and that matters for t-test, but not for Wilcoxon. E.g. we could change the values:

x <- c(10000, 11000, 12000)
y <- c(388.42733494, 531.10230857, 214.42573011)

wilcox.test(x, y, paired = TRUE )

and we still get p = 0.25.

You really need more data.