Wikipedia defines size of a test as the maximum probability of committing a type 1 error. And I'm very confused about how this is different than the significance level alpha. It seems to me the definition for both is P(rejecting null | null is true).

Is the size of a hypothesis test dependent on the type of test (z-test vs t-test), and does it depend on other parameters such as power, effect size?

Could you please explain this in the context of a t-test?


  • $\begingroup$ See also stats.stackexchange.com/questions/203212/… $\endgroup$ Commented Nov 24, 2020 at 11:08
  • $\begingroup$ How come it's so similar to my answer ?! XD $\endgroup$
    – JRC
    Commented Nov 24, 2020 at 13:07
  • $\begingroup$ @Peppershaker , I have fixed quite a few typos in my answer. Please let me know if it answers your query or not. $\endgroup$
    – JRC
    Commented Nov 24, 2020 at 13:08
  • $\begingroup$ @Kolmogorov, surely foresight on my part when I posted it four years ago ;-). On a more serious note, given the many duplicate questions that aren't always identified as such, duplicate answers are also to be expected every now and then, I guess. $\endgroup$ Commented Nov 24, 2020 at 13:19

1 Answer 1


Suppose, you've $X_1, X_2, \cdots , X_n \stackrel{\text{iid}}{\sim} N(\mu,1)$ , and you are testing $$H_0 : \mu \geqslant 0 \quad\text{against}\quad H_a : \mu < 0$$ Observe that the null hypothesis $H_0$ is a composite hypothesis, i.e. if $H_0$ is true, then for different values of $\mu~(\geqslant 0)$ , we shall obtain normal distributions with different parameters every time.

Therefore, for each possible value of $\mu~(\geqslant 0)$ , you get a different value of the type-I error (Size). For example, in this specific problem, there are uncountably many possible values of $\mu$ when $H_0$ is true. Now, the level of a test is defined to be the supremum of the set of all possible type-I errors. So, there may be infinitely many possible values of type-I errors. But the level will be a single real number, viz. the supremum of the set of all type-I errors.

Now, for the cases where $H_0$ is a simple null hypothesis, then the set of all possible type-I errors is just a singleton set. So, for that test, the level and type-I error are equal. e.g. suppose you're testing $$H_0 : \mu = 0 \quad\text{against}\quad H_a : \mu \neq 0$$ Then, the level and type-I error will indeed be equal.

Also, remember that the level or type-I errors of a test doesn't depend on its power, nor they are something special for different tests. They solely depend on the the type (Simple or Composite) of the null hypothesis $H_0$ , and also on the possible values that the concerned parameter may take if $H_0$ is true.

Hope this helps.

  • $\begingroup$ Thank you very much! $\endgroup$ Commented Nov 24, 2020 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.