# How must the null hypothesis be rejected for Nemenyi test to be allowed?

Suppose we have algorithms $A_k$ and $B_k$, $1\leq k\leq n$, and $\alpha = 0.05$.

Friedman test for the algorithms $A_k$ shows that the null hypothesis that all $A_k$s perform equally well, can be rejected. Suppose $p_A = 10^{-5}\ll\alpha$.

Friedman test for the algorithms $B_k$ shows that the null hypothesis that all $B_k$s perform equally well, cannot be rejected. Suppose $p_B = 0.4$.

My problem is that the $p$-value $p_{AB}$ from the Friedman test for $A_k$s and $B_k$s is greater than $\alpha$, hence I cannot apply Nemenyi post-hoc test if I blindly follow the procedure

• perform Friedman on algorithms
• if $H_0$ is rejected, proceed to Nemenyi on the same algorithms

Question: Are the results from Nemenyi test for all algorithms still valid?

I am not sure, but I would say yes because

• I have shown that already not all $A_k$s have the same performance (and the results would still be significant with $2p_A$, i.e., if I used Bonferroni correction for multiple-hypothesis testing)
• Nemenyi for $A_k$s and $B_k$s found two groups of algorithms, i.e., some differences.