1
$\begingroup$

I have data of program performance before and after an optimization of this program. The program was run indipendently several times before and after. I calculated the quotient of runtime per database size to have an estimate of the performance.

seconds/gigabyte: 1.52 0.21 0.66 0.71 0.02 0.33 0.27 0.40 0.20 0.00 0.00 0.00 0.00 2.58 0.01 0.00 1.64

arithmetic mean: 0.50

std: 7.383407471·10-1

The distribution is normal with a confidence of: 0.01% (Anderson-Darling normality test)

histogram of values before optimization, sec/gb


seconds/gigabyte:

0.11 0.01 0.01 0.00 0.21 0.05 0.17 0.00 0.00 0.00 0.14 0.06 0.15

arithmetic mean: 0.07

std: 7.626707459·10-2

The distribution is normal with a confidence of: 2.38% (Anderson-Darling normality test)

histogram of values after optimization, sec/gb

I wonder which mean would be the most suitable to compare these quotients and why? I chose the arithmetic because both database size and runtime vary per run of the program (or is this irrelevant? ^^).

I used an online anderson-darling-normality-test I think this indicates both are not normally distributed. Would a t-test be advisable?

I performed a Wilcoxon Rank Sum test to test for the null hypothesis that the performance is identical. I cannot reject this hypothesis because the p=0.1022. Is this correct?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.