# How to design a randomized test?

The manufacturers of a certain brand of electrical switches claim that at least $$80\%$$ of the boxes, containing 12 such switches each, will have no defective. Design a randomized test of exact size $$0.05$$ based on the number of boxes, among a random sample of 8 examined, containing at least one defective

My Attempt

Given probability of no defective in the sample is $$0.8.$$

So $$p = 0.2$$ is the probability of defective in the sample.

sample size is $$n=8.$$

Now the probability of at least one defective is

\begin{align}\textrm{P ( contains at least one defective in sample size of 8 boxes ) }&=\sum _{x=1 }^{8} \binom{n}{ x} p^x (1-p)^{n-x}\\&= 1 - \textrm{probability of no defective product in the sample}\\&= 1- (1-0.2)^{8-0}\\&= 0.8322\end{align}

My doubt

The question says design randomized test of size $$0.05 ,$$ I don't know how to "design "; if anyone can explain the requirement of the question, and if I am moving in the correct direction?

## 1 Answer

The claim is that no more than 20% of the boxes will contain a defective switch (let's call these "defective boxes"). Upon testing eight randomly selected boxes, what would it take to disprove that claim?

Consistent with this claim would be an observation of around $$20/100\times 8 = 1.6$$ defective boxes. Disproving it would require seeing an unusually large number of defective boxes. How large?

We know the chance of observing $$k \in \{0,1,\ldots, 8\}$$ defective boxes is Binomial,

$$\Pr(k) = \binom{8}{k} \left(\frac{20}{100}\right)^k \left(1 - \frac{20}{100}\right)^{n-k}.$$

Let's tabulate them.

0    1    2    3     4      5      6       7       8
Probability 0.17 0.34 0.29 0.15 0.046 0.0092 0.0011 8.2e-05 2.6e-06
Cumulative  1.00 0.83 0.50 0.20 0.056 0.0100 0.0012 8.4e-05 2.6e-06

"Cumulative" is the chance of observing $$k$$ or more defective boxes. It is computed by summing the values in the "Probability" line above and to the right. For example, the cumulative value of 0.056 for $$k=4$$ is $$0.056 = 0.046 + 0.0092 + 0.0011 + \ldots.$$

Evidently, the chance of observing $$k=4$$ or more defective boxes is 5.6% but the chance of observing $$k=5$$ or more defective boxes is only 1.0%. If we were not willing to use a randomized test, we would have to choose between these two options.

Formally, the test statistic is the count $$X$$ of defective boxes in the sample and our choice of a critical region for the test would be either $$\mathscr R_4 = \{4,5,6,7,8\}$$ or $$\mathscr R_5 = \{5,6,7,8\}.$$ This set notation refers to the values of the test statistic that we would consider to be sufficient evidence to refute the claim. The table shows

$$\Pr(X \in \mathscr R_4) = 0.056;\quad \Pr(X \in \mathscr R_5) = 0.010.$$

The difference between these critical regions is the outcome $$X=4.$$ Observing it would put you "on the fence." On one hand, $$4$$ is only in the critical region associated with a test level of 5.6%, which exceeds your target of 5%; on the other hand, it is nearly in the critical region with a level of 1%.

A randomized test procedure makes a random decision when the test statistic is on the fence. It does so by drawing a random uniform value $$U$$ and comparing it to a level $$p:$$ when $$U\le p,$$ we will reject the null hypothesis. (This amounts to flipping an "unfair coin" with probability $$p$$ of rejection.) Notice, from the table, that this makes the probability of rejection equal to

\begin{aligned} \Pr(\text{Reject }H_0) &= \Pr(X \in \mathscr R_4 \setminus \mathscr R_5)\Pr(U\le p) + \Pr(X \in \mathscr R_5) \\&= \Pr(X = 4)p + \Pr(X \ge 5) \\&= 0.046 p + 0.010. \end{aligned}

Because we want the randomized test to have level $$0.05,$$ we need to solve the resulting equation

$$0.05 = \Pr(\text{Reject }H_0) = 0.046 p + 0.010$$

which easily gives

$$p = \frac{0.05 - 0.010}{0.046} = 0.863$$

(this was computed with greater precision than shown in the rounded values I presented in the table).

To summarize, the test will be the following:

1. Obtain a random sample of $$8$$ boxes and test them exhaustively for defectives. Let $$X$$ be the number of boxes containing at least one defective switch.

2. Make a decision:

• If $$X \lt 4,$$ conclude the data are consistent with a claim that the manufacturing process produces defective boxes at a rate no greater than 20%.

• If $$X \gt 4,$$ conclude the claim is incorrect.

• If $$X = 4,$$ flip an unfair coin with probability $$p = 86.3\%$$ of heads. If it turns up heads, conclude the claim is incorrect. Otherwise, conclude the data are consistent with the claim.

In effect, you almost always conclude that observing $$X=4$$ is evidence against the claim, but a small percentage of the time this occurs (13.7%) you would be lenient and not reject the claim.

Our calculations have shown this procedure has exactly a 5% chance of rejecting the claim when the production rate of defective boxes is exactly 20%. Finally, argue that when the production rate is less than 20%, the chance of rejecting the claim is even less than 5%.

### References

For a more abstract general explanation of randomized tests, see https://stats.stackexchange.com/a/422171/919.

For a graphical account of finding critical regions, see https://stats.stackexchange.com/a/130772/919.

Jack Kiefer, An Introduction to Statistical Inference, has excellent explanations of randomized procedures at a general level requiring only basic mathematics.

• I am always impressed by your exceptional way of presenting the solution sir Commented Dec 21, 2022 at 12:59