0
$\begingroup$

Wikipedia defines size of a test as the maximum probability of committing a type 1 error. And I'm very confused about how this is different than the significance level alpha. It seems to me the definition for both is P(rejecting null | null is true).

Is the size of a hypothesis test dependent on the type of test (z-test vs t-test), and does it depend on other parameters such as power, effect size?

Could you please explain this in the context of a t-test?

Thanks,

$\endgroup$
  • $\begingroup$ See also stats.stackexchange.com/questions/203212/… $\endgroup$ – Christoph Hanck Nov 24 '20 at 11:08
  • $\begingroup$ How come it's so similar to my answer ?! XD $\endgroup$ – Kolmogorov Nov 24 '20 at 13:07
  • $\begingroup$ @Peppershaker , I have fixed quite a few typos in my answer. Please let me know if it answers your query or not. $\endgroup$ – Kolmogorov Nov 24 '20 at 13:08
  • $\begingroup$ @Kolmogorov, surely foresight on my part when I posted it four years ago ;-). On a more serious note, given the many duplicate questions that aren't always identified as such, duplicate answers are also to be expected every now and then, I guess. $\endgroup$ – Christoph Hanck Nov 24 '20 at 13:19
2
$\begingroup$

Suppose, you've $X_1, X_2, \cdots , X_n \stackrel{\text{iid}}{\sim} N(\mu,1)$ , and you are testing $$H_0 : \mu \geqslant 0 \quad\text{against}\quad H_a : \mu < 0$$ Observe that the null hypothesis $H_0$ is a composite hypothesis, i.e. if $H_0$ is true, then for different values of $\mu~(\geqslant 0)$ , we shall obtain normal distributions with different parameters every time.

Therefore, for each possible value of $\mu~(\geqslant 0)$ , you get a different value of the type-I error (Size). For example, in this specific problem, there are uncountably many possible values of $\mu$ when $H_0$ is true. Now, the level of a test is defined to be the supremum of the set of all possible type-I errors. So, there may be infinitely many possible values of type-I errors. But the level will be a single real number, viz. the supremum of the set of all type-I errors.

Now, for the cases where $H_0$ is a simple null hypothesis, then the set of all possible type-I errors is just a singleton set. So, for that test, the level and type-I error are equal. e.g. suppose you're testing $$H_0 : \mu = 0 \quad\text{against}\quad H_a : \mu \neq 0$$ Then, the level and type-I error will indeed be equal.


Also, remember that the level or type-I errors of a test doesn't depend on its power, nor they are something special for different tests. They solely depend on the the type (Simple or Composite) of the null hypothesis $H_0$ , and also on the possible values that the concerned parameter may take if $H_0$ is true.

Hope this helps.

$\endgroup$
  • $\begingroup$ Thank you very much! $\endgroup$ – Peppershaker Nov 24 '20 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.