What does it mean if Nemenyi test does not return that the performance of two methods are significantly different while the Wilcoxon test does? I want to compare the performance of $N$ methods on $m$ datasets. I performed the Friedman test and after that the Nemenyi test.
The Nemenyi test is though not powerful enough to conclude that there is a significant difference between the performance of the best performing method and the second performing method.
I then used the Wilcoxon method to compare the performance of these two models, and Wilcoxon method indeed returns that the difference in their performance is significant.
What can I conclude? Can I conclude that the difference in performance of the two models is statistically significant?
 A: Per Wikipedia on Wilcoxon signed-rank test:

Assumptions

*

*Data are paired and come from the same population.


*Each pair is chosen randomly and independently[citation needed].


*The data are measured on at least an interval scale when, as is usual, within-pair differences are calculated to perform the test (though it does suffice that within-pair comparisons are on an ordinal scale).

Now, the Friedman test, a commonly employed non-parametric test for complete block designs is related to the non-parametric Durbin test as it  reduces to the Friedman test for complete block design scenarios. The cited underlying assumptions per a source are:


*

*The b blocks are mutually independent. That means the results within one block do not affect the results within other blocks.


*The data can be meaningfully ranked (i.e., the data have at least an ordinal scale).

The Nemenyi post-hoc test, however, has been discussed previously on this forum here, and the appropriate underlying assumptions are those that apply for the use of the studentized range distribution. Per Wikipedia, these are:

Assumptions

*

*The observations being tested are independent within and among the groups.


*The groups associated with each mean in the test are normally distributed.


*There is equal within-group variance across the groups associated with each mean in the test (homogeneity of variance).

Note, the Nemenyi test has an implicit assumption normality of means. This assumption is absent elsewhere, as such, I would more likely suspect, that there is a non-normality issue. Also, per the references cited, be mindful of the 'family of test' significance level adjustment.
