# What is the size of a t-test?

Wikipedia defines size of a test as the maximum probability of committing a type 1 error. And I'm very confused about how this is different than the significance level alpha. It seems to me the definition for both is P(rejecting null | null is true).

Is the size of a hypothesis test dependent on the type of test (z-test vs t-test), and does it depend on other parameters such as power, effect size?

Could you please explain this in the context of a t-test?

Thanks,

• – Christoph Hanck Nov 24 '20 at 11:08
• How come it's so similar to my answer ?! XD – Kolmogorov Nov 24 '20 at 13:07
• @Peppershaker , I have fixed quite a few typos in my answer. Please let me know if it answers your query or not. – Kolmogorov Nov 24 '20 at 13:08
• @Kolmogorov, surely foresight on my part when I posted it four years ago ;-). On a more serious note, given the many duplicate questions that aren't always identified as such, duplicate answers are also to be expected every now and then, I guess. – Christoph Hanck Nov 24 '20 at 13:19

Suppose, you've $$X_1, X_2, \cdots , X_n \stackrel{\text{iid}}{\sim} N(\mu,1)$$ , and you are testing $$H_0 : \mu \geqslant 0 \quad\text{against}\quad H_a : \mu < 0$$ Observe that the null hypothesis $$H_0$$ is a composite hypothesis, i.e. if $$H_0$$ is true, then for different values of $$\mu~(\geqslant 0)$$ , we shall obtain normal distributions with different parameters every time.
Therefore, for each possible value of $$\mu~(\geqslant 0)$$ , you get a different value of the type-I error (Size). For example, in this specific problem, there are uncountably many possible values of $$\mu$$ when $$H_0$$ is true. Now, the level of a test is defined to be the supremum of the set of all possible type-I errors. So, there may be infinitely many possible values of type-I errors. But the level will be a single real number, viz. the supremum of the set of all type-I errors.
Now, for the cases where $$H_0$$ is a simple null hypothesis, then the set of all possible type-I errors is just a singleton set. So, for that test, the level and type-I error are equal. e.g. suppose you're testing $$H_0 : \mu = 0 \quad\text{against}\quad H_a : \mu \neq 0$$ Then, the level and type-I error will indeed be equal.
Also, remember that the level or type-I errors of a test doesn't depend on its power, nor they are something special for different tests. They solely depend on the the type (Simple or Composite) of the null hypothesis $$H_0$$ , and also on the possible values that the concerned parameter may take if $$H_0$$ is true.