I have two algorithms A and B which have to compute a solution to some problem. Each solution is given some objective value which indicates the quality of the solution. I need to perform a Wilcoxon Signed-Rank Test to test whether there is any evidence that these two algorithms perform statistically significantly different from one another.

I have performed 12 trials of each algorithm and tabulated the objective values from solutions found during each trial. A smaller objective value is better.

 A   B
878 890
872 888
865 879
877 874
872 870
890 886
873 871
887 879
868 873
888 882
878 881

I am confused about a few details of performing this test.

  • Should I do a one-tailed or two-tailed test?

  • I'm not sure what my null hypothesis is. What should it be, given I want to find out whether algorithm A and B perform significantly different from one another?

  • If $\ p$-value $> 0.05$, what does this mean?

  • If $\ p$-value $< 0.05$, what does this mean?

  • 1
    $\begingroup$ Is this homework? Whether or not it is, it seems to me that the "self-study" flag would be appropriate... $\endgroup$ – jbowman May 13 '19 at 21:48
  • 1
    $\begingroup$ 1. There are many questions that relate to interpreting p-values already on site; please try a site-search (but in any case, you should begin with basic resources that are easy to find with an internet search like the wikipedia page on p-value). 2. "Should I do a one-tailed or two-tailed test?" ... this is not a question for us to guess at; you need to clarify your question enough that it's clear whether you're after only one direction of difference or whether a difference in either direction is relevant to you. It doesn't look form what you wrote like you want a one-tailed test $\endgroup$ – Glen_b -Reinstate Monica May 14 '19 at 4:49
  • 1
    $\begingroup$ 3. Note that hypotheses are about populations, not samples, and the word "significantly" (or variations on it) don't belong in them. $\endgroup$ – Glen_b -Reinstate Monica May 14 '19 at 4:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.