Q: Determine Confidence level based on measurements

Question

Its been a while since I last did some statistics so forgive me If I am bit rusty. Please point out any obvious errors or omissions.

Thanks for any advice given.

Wally4u

Due to some new regulations we need to measure and prove that a manufacturing process is reliable. We originally specified the area ± 5 % in the specifications (a bit stricter than required). And we did that with spot measurements (AQL measurements)

Part of the regulations give the following rule:

The new regulations allow for continued spot measurements IF the following rule is met:

Under normal test conditions, the results should produce an overall uncertainty in the determination of area (at the 95 % confidence level) of ± 10 %.

So now I'm trying to figure out based on old measurement data if we meet the new rule AND our specification.

I've inputted the source data in excel as a list.

4.800 4.930 4.490 4.830 5.100 4.650 4.540 4.940 5.150 4.670 5.040 4.720 4.430 4.540 4.710 4.720 4.860 4.610 4.770 4.810 5.210 4.890 4.640 4.970 5.070 5.260 5.220 4.980 5.000 5.100 5.210 5.120 5.310 5.200 5.350 5.110 5.000 5.270 4.880

I use excel to calculate the following:

Average =AVERAGE(J5:J43) --> 4.925641026

standard deviation =STDEV.S(J5:J43) --> 0.249818828

confidence =1-CONFIDENCE.NORM(0.05,STDEVRESULT,COUNT(J5:J43)) --> 0.921595506

Uncertainty TYPE A =J48/SQRT(COUNT(J5:J43)) --> 0.040003028

What we used to claim is that the area measured is 5mm ± 5 %

Q1. Can I prove that we follow the new guidelines based on the above approach? In other words, can I use old measurements to prove that within 95% confidence that our production is within ± 5 % of stated 5mm?

edit response Joel

R1. Makes sense to determine the uncertainty of the measurement on a single unit. Main reason why I wanted to know if I can use the old data since we already have the data.

Q2. To calculate the confidence (as I did above) level can / should I use all data (in this case J5:J43) or should I use a smaller dataset.

edit response Joel

R2. The origin of this question refers to how many sample sizes should be sufficient. The referenced NPL document states 20 sample is a bit better than 10 sample but 50 samples is not really improvement over 20 samples.

Q3. The standard deviation is based on the measurements. In the Confidence.NORM function should I use the value calculated or 5% maximum allowed deviation? I'm doubting myself here which one to use.

The Excel CONFIDENCE.NORM function syntax has the following arguments:

CONFIDENCE.NORM(alpha,standard_dev,size)

Alpha     Required. The significance level used to compute the confidence level. The confidence level equals 100*(1 - alpha)%, or in other words, an alpha of 0.05 indicates a 95 percent confidence level.

Standard_dev     Required. The population standard deviation for the data range and is assumed to be known.

Size     Required. The sample size.

edit response Joel

R3.Thanks, I will take a look at the CONFIDENCE.T function. I also noted that in my example I used 1-CONFIDENCE.NORM, which I believe now is incorrect. Re-reading the result of function it give the confidence interval. The 1 serves no purpose in this case.

edit response Joel

Q4. Please correct me if the following is incorrect: (Assuming the measurements listed above are from one sample unit (hypothetically, since I need to redo the measurements) )

a. I can use the STDEV.S as an input variable for the CONFIDENCE.T (calculated over all measurements) =STDEV.S(J5:J43) --> 0.249818828

b. Calculating the =CONFIDENCE.T(0.05,0.249818828,39) results in the confidence interval of 0.080981896. Meaning, I can say that with 95% confidence that the range is from (Average of samples - 0.080981896) AND (Average of samples + 0.080981896)

c. the uncertainty of the measurement (assuming measurements are done on the same sample unit and same method) Will be u = =STDEV.S(J5:J43) / SQRT(39) --> 0.040003028 meaning 4% uncertainty of the measurement.

As a base reference for the above I used the NPL A Beginner's Guide to Uncertainty of Measurement https://www.wmo.int/pages/prog/gcos/documents/gruanmanuals/UK_NPL/mgpg11.pdf

Answer 1

Q1: My interpretation of the guideline you quote is that the measurement process must have an uncertainty of $\pm$ 10%. Typically this is estimated by taking multiple measurements on the same unit. I don't think you can use your data set to show that you meet the guideline for continued spot checking.

Q2: I don't see what you would gain from a smaller sample size. However, right now your biggest problem isn't the amount of data that you have, but the type of data.

Q3: The interval computed by CONFIDENCE.NORM assumes you know the population standard deviation precisely. In this case, you only have an estimate of the population standard deviation, which means you'll be underestimating the uncertainty of your measurement process. You should use the CONFIDENCE.T function instead.

Edit to respond to additional questions

Q4. Please correct me if the following is incorrect: (Assuming the measurements listed above are from one sample unit (hypothetically, since I need to redo the measurements) )

a. I can use the STDEV.S as an input variable for the CONFIDENCE.T (calculated over all measurements) =STDEV.S(J5:J43) --> 0.249818828

Yes. The documentation for this function isn't particularly helpful, but as far as I can tell, CONFIDENCE.T computes the value $t_{1 - \alpha / 2, n-1} s / \sqrt{n}$ which would be half the width of the 95% (two-sided) confidence interval for the mean of your 39 measurements.

b. Calculating the =CONFIDENCE.T(0.05,0.249818828,39) results in the confidence interval of 0.080981896. Meaning, I can say that with 95% confidence that the range is from (Average of samples - 0.080981896) AND (Average of samples + 0.080981896)

I'd be careful with the word 'range'. The confidence interval you just calculated describes where we think the mean would lie if we kept taking additional measurements. It's quite possible that additional measurements would result in a mean outside of the confidence interval.

c. the uncertainty of the measurement (assuming measurements are done on the same sample unit and same method) Will be u = =STDEV.S(J5:J43) / SQRT(39) --> 0.040003028 meaning 4% uncertainty of the measurement.

This is the standard error of the mean of the measurements, which is one way to express the uncertainty. However, based on its wording, the guideline seems to imply that uncertainty should be expressed as the width of the 95% confidence interval of the mean (i.e. the result of Q4b).