# Find probability of rejecting a true null hypothesis

A binomial experiment tests $H_0$ : p = 0.5 against p $\neq$ 0.5 using significance level 0.05. Only 5 observations are available. Show that the probability to reject a true null hypothesis is 1/16.

The steps for the solution:

$H_0$ : p = 0.5

$H_a$: p $\neq$ 0.5

$N$ = 5

$\alpha$ = 0.05

Pr(Type I error) = Pr(reject $H_0$ when $H_0$ is true)

The formula implemented from Hypothesis testing, page 12

$\sum_{x=1}^5 b(x, n = 5, p = 0.5) = \sum_{x=1}^5 \binom{5}{n} 0.5^x0.95^{5-x}$

but it doesn't get the answer 1 / 16.

Any suggestion what's wrong ?

• Is this an exact binomial test or are you using the normal approximation to the sample proportion? Commented May 29, 2018 at 15:32
– Bor
Commented Jun 1, 2018 at 8:17

For calculating the probability of a Type I Error, we start with: $$$$\label{eql} \begin{split} \text{Pr}(\text{Type I Error}) & = \text{Pr}(\text{reject }H_0 | H_0 \text{ is true}) \\ & = \text{Pr}(\text{reject }H_0 | p=.5, n=5) \end{split}$$$$

The probability mass function $\text{Pr}(X=x)=\binom{5}{x}.5^x .5^{5-x}$ (note that your pmf incorrectly uses $1-p=.95$) for a binomial random variable $X$ given our $H_0$ ($p=.5,n=5$) is: $$\begin{split} \text{Pr}(X=0) = \frac{1}{32} = .03125 \\ \text{Pr}(X=1) = \frac{5}{32} = .15625 \\ \text{Pr}(X=2) = \frac{5}{16} = .31250 \\ \text{Pr}(X=3) = \frac{5}{16} = .31250 \\ \text{Pr}(X=4) = \frac{5}{32} = .15625 \\ \text{Pr}(X=5) = \frac{1}{32} = .03125 \end{split}$$

Noting above that only $\text{Pr}(X=0)$ and $\text{Pr}(X=5)$ are below our $\alpha=.05$ threshold, and therefore that $H_0$ may only be rejected if a sample results in $X=0$ or $X=5$, we can move forward as follows:

$$$$\label{eql1} \begin{split} \text{Pr}(\text{Type I Error}) & = \text{Pr}(\text{reject }H_0 | p=.5,n=5) \\ & = \text{Pr}(X=0| p=.5,n=5) + \text{Pr}(X=5| p=.5,n=5) \\ & =2\cdot.03125=.0625=\frac{1}{16} \end{split}$$$$

• But this doesn't meet the goal of having the false alarm probability being at most $0.05$; the false alarm probability is $0.0625 > 0.05$. Commented May 29, 2018 at 20:45
• is missing the 0 in .5 some kind of convention here ?
– Bor
Commented May 30, 2018 at 7:48
• also why are 6 observations made(0-5) ?
– Bor
Commented May 30, 2018 at 11:59
• 1) Not really - some people do it, some don't. It's a matter of preference, and I find it to be more easily readable. 2) A total of 0, 1, 2, 3, 4, or 5 successes can possibly be observed, because there are 5 observations in the experiment. An example would be being able to flip anywhere from 0 to 5 heads when flipping n=5 times. Commented May 30, 2018 at 12:09
• In other words, there are 6 possible outcomes of the experiment when you make 5 observations. Commented May 30, 2018 at 12:18

I have no idea what the significance level is doing in this hypothesis test or what its meaning might be, but stripped of all the malarkey and statistical verbiage, on 5 tosses of a fair coin, the probability of tossing 5 heads or 5 tails is $\displaystyle \frac{1}{32}+ \frac{1}{32}= \frac{1}{16}$. Any reasonable test of the hypothesis $H_0$ that $p=0.5$ versus the hypothesis $H_1$ that $p\neq 0.5$ must reject this event (all heads or all tails) giving a false alarm probability of $\frac{1}{16}=0.0625$ on the reasonable assumption that the only observation that causes rejection of $H_0$ is the all heads or all tails event. If you insist that the false alarm probability should not exceed 0.05, then you either need to use an asymmetric test such as rejecting $H_0$ when you observe 5 heads but not rejecting $H_0$ when you observe 5 tails, or, more simply, never reject $H_0$ thereby achieving a false alarm probability of $0$ which, when I last checked, is smaller than the maximum allowable false alarm probability of $0.05$.

• No more clarifications are given on the task, just to prove it's 1/16
– Bor
Commented May 30, 2018 at 7:15