What kind of statistical test should I use to determine if a manufacturing process leads to significantly less scrap rate?

Question

Assume in a factory, each day approximately 10,000 products are made. The scrap rate ranges from 0.1% to 1%. You want to try a new process or test a different material, and you make a test run of 500 products, and obtained a scrap rate of x%.

What kind of test can one use to determine if the difference in scrap rate is statistically significant? Will a T test work? Suppose for the past 30 days, you collected scrap rate for each day of normal production and it's a normal distribution.

I'm not sure if T test is valid due to differing sample size of each "trial". Not only is 500 much different from 10,000, but even during the 30 days of normal production, some day you might make more than 10,000, some day you might make less.

Another test I'm thinking about is a Chi-squared test. Please advise.

Thanks

Answer 1

Here is a suggestion how you might start thinking about this problem.

Let $p$ be the scrap rate. Of the possibilities suggested in the statement of your problem, maybe you want to test $H_0: p = 0.01$ against $H_a: p < 0.01$ at about the 5% level.

Under the null hypothesis, the number of junk items in $n = 500$ is $X \sim \mathsf{Binom}(n=500,p=0.01).$ So under the null hypothesis, $P(X \le 1) = 0.04 < 0.05.$ [Computation in R.]

pbinom(1, 500, .01)
[1] 0.03975474

So getting 0 or 1 junk item in 500 would be taken as evidence that the junk rate with the new process is below 1%.

If you have data from several weeks with more than $n = 500$ items produced using the new process, then other tests would be possible and it might be convenient to use the normal approximation to a binomial probability.