I have two implementations of a genetic algorithm which are supposed to behave equivalently. However due to technical restrictions which cannot be resolved their output is not exactly the same, given the same input.
Still I'd like to show that there is no significant performance difference.
I have 20 runs with the same configuration for each of the two algorithms, using different initial random number seeds. For each run and generation the
minimum error fitness of the best individual in the population was recorded. The algorithm employs an elite-preserving mechanism, so the fitness of the best individual is monotonically decreasing. A run consists of 1000 generations, so I have 1000 values per run. I cannot get more data, as the calculations are very expensive.
Which test should I employ? An easy way would probably be to only compare the error in the final generations (again, which test would I use here)? But one might also think about comparing the convergence behaviour in general.