3
$\begingroup$

My book mentions

$$\textrm{Probability of Type I Error}\le \textrm{Level of Significance}= \alpha$$

Now, I bore in mind that $\alpha$, the level of significance is described by $$\mathrm P(t\in \omega|H_0)= \alpha$$ where $\omega = \textrm{critical region}\,.$

I could get the sense of probability of Type I error equals $\alpha$ but when it is less than $\alpha\;?$

I'm not getting that.

Is it a textbook-erratum? If not, when does this case arise?

|cite|improve this question
$\endgroup$
  • $\begingroup$ It should be $\omega=$critical region, no? $\endgroup$ – Christoph Hanck Mar 23 '16 at 12:26
  • 2
    $\begingroup$ It could happen with discrete distributions, e.g. with hypothesis tests based on Binomial random variables $\endgroup$ – user83346 Mar 23 '16 at 12:32
  • $\begingroup$ @fcop: Would I be wrong if I say answers are not allowed in comment ;P Please make it an answer; I would be grateful. $\endgroup$ – user74724 Mar 23 '16 at 12:37
  • $\begingroup$ @ChristophHanck: Sorry; fixing it. $\endgroup$ – user74724 Mar 23 '16 at 12:37
  • $\begingroup$ To complement @fcop's comment and answer: you have to distinguish the nominal (desired, or "target") significance level and the effective significance level. $\endgroup$ – Stéphane Laurent Mar 23 '16 at 14:16
3
$\begingroup$

It is not a typo and for example arises when testing composite hypotheses, such as a one-sided test of the mean of a distribution, as in $H_0:\mu\leq0$ against $H_0:\mu>0$. We choose the critical value (equivalently, $\omega$) so that the probability of a type-I error is still just $\alpha$ even when $\mu$ is very close to being in the set specified by $H_1$, i.e., when $\mu=0$, i.e., when the probability of a type-I error is highest.

Now if, say, the true $\mu$ equals $-5$, the probability of obtaining a sample such that $t\in\omega$ will be much smaller. For example, suppose we sample from a $N(\mu,1)$ distribution. Then, our t-statistic for known $\sigma$ and testing against positive $\mu$ would simply be $t=\sqrt{n}\bar{x}$ and follow a $N(\mu,1)$ distribution, too.

The plot below shows the probability of rejecting $H_0$, i.e., the power function. The region to the left of the vertical dashes is where the null is true, and you can see that the rejection probability is .05 only at $\mu=0$.

enter image description here

mu <- seq(-3,3,by=.02)
plot(mu,1-pnorm(1.645,mean=mu),type="l",lwd=2,col="sienna3")
abline(h=.05,lty=2)
abline(v=.0,lty=2)
|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks, sir, for the answer. So, level of significance is a much broader term than probability of Type I error, I guess? $\endgroup$ – user74724 Mar 23 '16 at 12:00
  • 3
    $\begingroup$ It is the supremum over the rejection probabilities taken over all parameters in the set specified by $H_0$. $\endgroup$ – Christoph Hanck Mar 23 '16 at 12:11
2
$\begingroup$

The probability of a type I error may be stricly smaller than the significance level $\alpha$ for discrete distributions. E.g. if you want to test whether a coin is upward biased i.e. $H_1: p > 0.5$ and we test this against $H_0: p \le 0.5$.

Then the critical region is in the right queue, assume that we observe 10 tosses. Choosing $\alpha=0.05$ the critical region is $\omega = \{9,10\}$ and $P(t\in\omega)=0.0107$

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ @fcop (+1) you surely mean $H_0: \le 0.5$ $\endgroup$ – peuhp Mar 23 '16 at 20:11
  • $\begingroup$ @peuhp: you are right, I edited the answer $\endgroup$ – user83346 Jun 20 '16 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy