Having recently run an experiment, I have been left with a dataset that I don't quite know how best to handle, I think simply due to the number of independent variables to consider.
I have implemented 4 new approaches to solve a problem, which I wish to compare to an existing approach and to each other, based on their execution time, which prior to the experiment is expected to be an improvement.
To compare these approaches, they are each tested with a set of 6 case studies. Each combination of approach-casestudy is repeated 30 times, to ensure the results are representative.
All of the above is repeated for two different libraries which are used as part of the approach.
In total, I am left with a total of 1800 rows (calculated as...
= (Approaches * Case Studies * Trials * Libraries) = 5 * 6 * 30 * 2 = 1800
I believe I could validly compare the results of two approaches for a specific library and case study using the non-parametric Wilcoxon rank-sum test. However, I don't know of a testing approach I could use to determine, overall, whether one approach is better than another, or which (for the given case studies and libraries) is 'best'.
Is there some approach I could use to validly summarise the results, and therefore conclude to some level of significance which is 'best'?
Thanks, and apologies for any missing important details- I will edit whenever necessary!