Excel T-Test “Set X is better than Set Y by ___%” Coming from this post, but their question wasn't fully answered.
I understand how the t-test works – if I have two sets of data and want to see if one is better (or worse) than the other, I would say ttest(setx, sety, 1, 2) for a type 2, and that would tell me whether or not one set is better/worse than the other. What I don't understand is how to say "Set X is better than set Y by __% with statistical significance." My best guess would be using ttest(setx, sety*.2, 1, 2), and if that still gives me a small enough p-value, then that .2 would turn into "Set X is better than set Y by 80%." Am I thinking that through correctly?
 A: There are two parts to your question. The first is that you want to use a one-tailed t-test so that you are testing setx is "better" (the mean of the population from which setx is a sample is larger). That is easily done with the t.test() function by setting the second parameter to 1 for one tail. Thus t.test(setx,sety,1,2) will perform a t-test for the mean of the population x being larger than the mean of population y, assuming equal variances and the conventional nil-null hypothesis.
It sounds to me like you want to set the null hypothesis for the test to be something other than zero. As excel doesn't allow specification of a value for the null hypothesis you will need to subtract the null hypothesised value from one of your sets of values before implementing the test. If you want to test if setx exceeds sety by 10% then you have to subtract 10% from setx and test whether the adjusted set exceeds sety by 0%, which is the nil-null hypothesis.
(Any reasonable statistical software should require specification of the null hypothesis in my opinion. To not do so is to allow users to remain ignorant of the actual nature of the test being used.)
