What statistical signifcance test should I use?

Question

I would like to compare the averages of two list of numbers, and see whether one average (from one "system" output) is higher than the average of another in a statistically significant way.

I already know which system I want to have a higher average. It does indeed have a higher average (pfjew).

However, what is the right statistical significance to test whether the population average for my system is indeed higher?

I looked at a variety of tests, such as t-test or Wilcoxon nonparametric test, and they seem to have a null hypothesis that says the systems are identical and that the H1 hypothesis is that the means are not identical. That doesn't quite sound right to me.

I think I need:

H0: other system is better or equal to my system
H1: my system is better

I suppose I want a one-tailed test, but can someone direct me better the exact name of the test?

Telling me for example what I want to do from here:

https://www.mathworks.com/help/stats/ttest.html

would be best (I am fine also with Wilcoxon nonparametric test)

Answer 1

One virtue of significance testing is the ability to conclude 'data compatible with no difference [two-sided question] / no superiority [one-sided question]'. In your situation [the one-sided one], if the observed superiority in the data is "significant," i.e., NOT well compatible with zero trrue superiority, then it is even less compatible with a negative superiority (i.e., inferiority) of whatever magnitude one might hypothesize. So you conclude: Superior!

The Ho vs. H1 of your question do fit into what I have just written.

(But note that in some situations the "even less" argument doesn't work because the model structure is non-monotonic. For instance, if a slight inferiority is accompanied by a big increase in data variance. - Here we typically have cum. distr. curves of the data average that cross, and likelihood graphs that are non-monotonic.)