0
$\begingroup$

I would like to compare the averages of two list of numbers, and see whether one average (from one "system" output) is higher than the average of another in a statistically significant way.

I already know which system I want to have a higher average. It does indeed have a higher average (pfjew).

However, what is the right statistical significance to test whether the population average for my system is indeed higher?

I looked at a variety of tests, such as t-test or Wilcoxon nonparametric test, and they seem to have a null hypothesis that says the systems are identical and that the H1 hypothesis is that the means are not identical. That doesn't quite sound right to me.

I think I need:

H0: other system is better or equal to my system
H1: my system is better

I suppose I want a one-tailed test, but can someone direct me better the exact name of the test?

Telling me for example what I want to do from here:

https://www.mathworks.com/help/stats/ttest.html

would be best (I am fine also with Wilcoxon nonparametric test)

$\endgroup$
0
$\begingroup$

One virtue of significance testing is the ability to conclude 'data compatible with no difference [two-sided question] / no superiority [one-sided question]'. In your situation [the one-sided one], if the observed superiority in the data is "significant," i.e., NOT well compatible with zero trrue superiority, then it is even less compatible with a negative superiority (i.e., inferiority) of whatever magnitude one might hypothesize. So you conclude: Superior!

The Ho vs. H1 of your question do fit into what I have just written.

(But note that in some situations the "even less" argument doesn't work because the model structure is non-monotonic. For instance, if a slight inferiority is accompanied by a big increase in data variance. - Here we typically have cum. distr. curves of the data average that cross, and likelihood graphs that are non-monotonic.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.