I have data of program performance before and after an optimization of this program. The program was run indipendently several times before and after. I calculated the quotient of runtime per database size to have an estimate of the performance.
seconds/gigabyte: 1.52 0.21 0.66 0.71 0.02 0.33 0.27 0.40 0.20 0.00 0.00 0.00 0.00 2.58 0.01 0.00 1.64
arithmetic mean: 0.50
The distribution is normal with a confidence of: 0.01% (Anderson-Darling normality test)
0.11 0.01 0.01 0.00 0.21 0.05 0.17 0.00 0.00 0.00 0.14 0.06 0.15
arithmetic mean: 0.07
The distribution is normal with a confidence of: 2.38% (Anderson-Darling normality test)
I wonder which mean would be the most suitable to compare these quotients and why? I chose the arithmetic because both database size and runtime vary per run of the program (or is this irrelevant? ^^).
I used an online anderson-darling-normality-test I think this indicates both are not normally distributed. Would a t-test be advisable?
I performed a Wilcoxon Rank Sum test to test for the null hypothesis that the performance is identical. I cannot reject this hypothesis because the p=0.1022. Is this correct?