My understanding is that the Friedman test is usually performed on unreplicated blocked data. A matrix containing such data would look as follows:

                 Group 1    Group 2    Group 3    ...    Group k
Block 1
Block 2
Block 3
Block n

In a situation where each "group" is different classification algorithm and each "block" is a different classification dataset, is it safe to perform Kruskal-Wallis test on the data by assuming that the "blocks" are simply replicates? That is,

                 Group 1    Group 2    Group 3    ...    Group k
Replicate 1
Replicate 2
Replicate 3
Replicate n
  • $\begingroup$ Why do you want to do Kruskal-Wallis test if Friedman test is recommended for this situation? $\endgroup$ – rnso Apr 15 '15 at 12:59
  • $\begingroup$ Because the conditions of the experiment did not change from one "block" to another. The classification algorithms use the same parameters in all "blocks". The only thing that varies from one "block" to the next is the dataset. Each "block" is a different fold of n-fold cross-validation. I hope I am making sense. $\endgroup$ – Prometheus Apr 15 '15 at 13:07
  • $\begingroup$ I think if you can decide to merge all replicates of a group, then Kruskal-Wallis can be used. $\endgroup$ – rnso Apr 15 '15 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.