3
$\begingroup$

I want to compare the speed of two algorithms. However I'm having troubles interpreting the results. Is my approach correct?

Null hypothesis: The algorithms have same speed

Alternative hypothesis: Algorithm Y is faster.

my data:

algorithm_x = np.array(
    [1014, 1007, 998, 1040, 999, 1030, 980, 1010, 940, 1030, 1000, 990, 1000, 995, 1020, 990, 1040, 1020, 1015, 940])
algorithm_y = np.array(
    [980, 995, 960, 1050, 970, 1010, 1005, 1020, 950, 1000, 1025, 970, 965, 980, 1015, 985, 1010, 995, 990, 955])

using scipy I calculated the p-value:

pvalue = scipy.stats.ttest_rel(algorithm_x, algorithm_y, alternative="greater").pvalue

and I got the result: $pvalue = 0.011638$

I was thinking the following:

Because the pvalue is less than our confidence interval 95% (0.05) we reject null hypothesis, thus the alternative hypothesis is true. So algorithm Y is indeed faster than algorithm X.

Is my thinking correct?

$\endgroup$

1 Answer 1

6
$\begingroup$

Your formulation should be corrected, but your conclusion may be okay, although ...

It is wrong to say that "the p value is less than the confidence interval". Change it into: "The p value is less than the chosen value of $\alpha$ (0.05)".

It is wrong to say that Y is indeed faster than X. It's better to say: "The test result supports the alternative hypothesis that Y is faster than X". Statistical tests only can make an idea more plausible, but never proof it.

Finally: are you sure you can test one-sided? Is there a good reason why Y could not be slower than X, on the average? Looking at your data, Y sometimes is slower, so I would suggest to test two-sided.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.