I have run a single algorithm on a 50 problems with 3 different values for a parameter(tiebreak). The algorithm's job is to rank candidate values for a parameter(split) that will be fed into another algorithm. However, I would like to perform a rank correlation test to determine whether there is statistically significant evidence to suggest different rankings against the 3 different tiebreaks. What test would achieve this? I have considered tau's test but the problem is that I am using the same 50 problems accross the tests hence a higher chance of a type I error. Is there a way to adjust for this or even better still a nonparameteric test that is designed for multiple comparisons.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.