When is probability of type-I error less than the level of significance?

Question

My book mentions

$$\textrm{Probability of Type I Error}\le \textrm{Level of Significance}= \alpha$$

Now, I bore in mind that $\alpha$, the level of significance is described by $$\mathrm P(t\in \omega|H_0)= \alpha$$ where $\omega = \textrm{critical region}\,.$

I could get the sense of probability of Type I error equals $\alpha$ but when it is less than $\alpha\;?$

I'm not getting that.

Is it a textbook-erratum? If not, when does this case arise?

Answer 1

It is not a typo and for example arises when testing composite hypotheses, such as a one-sided test of the mean of a distribution, as in $H_0:\mu\leq0$ against $H_0:\mu>0$. We choose the critical value (equivalently, $\omega$) so that the probability of a type-I error is still just $\alpha$ even when $\mu$ is very close to being in the set specified by $H_1$, i.e., when $\mu=0$, i.e., when the probability of a type-I error is highest.

Now if, say, the true $\mu$ equals $-5$, the probability of obtaining a sample such that $t\in\omega$ will be much smaller. For example, suppose we sample from a $N(\mu,1)$ distribution. Then, our t-statistic for known $\sigma$ and testing against positive $\mu$ would simply be $t=\sqrt{n}\bar{x}$ and follow a $N(\mu,1)$ distribution, too.

The plot below shows the probability of rejecting $H_0$, i.e., the power function. The region to the left of the vertical dashes is where the null is true, and you can see that the rejection probability is .05 only at $\mu=0$.

mu <- seq(-3,3,by=.02)
plot(mu,1-pnorm(1.645,mean=mu),type="l",lwd=2,col="sienna3")
abline(h=.05,lty=2)
abline(v=.0,lty=2)

Answer 2

The probability of a type I error may be stricly smaller than the significance level $\alpha$ for discrete distributions. E.g. if you want to test whether a coin is upward biased i.e. $H_1: p > 0.5$ and we test this against $H_0: p \le 0.5$.

Then the critical region is in the right queue, assume that we observe 10 tosses. Choosing $\alpha=0.05$ the critical region is $\omega = \{9,10\}$ and $P(t\in\omega)=0.0107$