Statistical Sampling for Tasks With Different Complexities

Question

We have contractors submitting tasks of different complexities (High/Medium/Low) and we would like to audit them. Considering the auditing resource availability, we have to sample the submitted tasks and perform auditing.

I am planning to adopt "ISO 2859-1: Sampling procedures for inspection by attributes" and use the corresponding tables for sample size & AQL selection. My understanding is that choosing the level (Level 1, 2 or 3) will determine the sample size & AQL(1,1.5,2.5 etc.) will determine the acceptance/rejection limits.

From a quick calculation, choosing Level 1 for all complexity levels will be within our resource capacity constraint. However, I am worried whether it increases the risk for 'High' complexity tasks by auditing very low (~5%) of them. On the other side, choosing Level 2 will result in auditing decent number of' High' complexity tasks (>10%) but the over all audits required will be beyond our resource capacity.

I am wondering whether it makes sense to go for Level 2 inspection for 'High' complexity tasks and Level 1 inspection for 'Medium/Low' complexity tasks. Similarly opt for a stricter AQL (say 1%) for 'High' & a liberal AQL (2.55 or 5%) for 'Medium/Low' ones.

Please let me know your thoughts and suggestions. Thank you

Answer 1

The inspection level you choose is entirely dependent on risk appetite, not statistics. It's barely a parameter that depends on your business constraints, i.e., how sure you want to be about your results, rather than a statistical property for which "right" and "wrong" values exist.

Your understanding is correct: both the inspection level and AQL are independent parameters of your sampling scheme that in turn determine the acceptance/rejection. Indeed you find two pairs of tables in the standard: first you choose the sampling code letter based on the lot size and your chosen inspection level ("Sample size code letters"), and then from that code letter you get a fixed sample size and acceptance/rejection numbers based on your chosen AQL ("Master tables").

It can make sense to have different inspection levels based on some parameters of the items to review, such as task complexity or any other property. This should be based on a reason to justify the choice in business terms, most likely in case you get audited in turn. The underlying reasoning here could be that you assume a higher likelihood of errors in more complex tasks, and you want to be more sure of their estimate -- hence a higher inspection level. Even better if you can tie this to inherent risk, i.e., errors in complex tasks have a larger business impact than errors in simpler tasks.

Note that sampling few items is not very much a concern as long as the standard is used for what's intended for, i.e., recurring lot-by-lot inspection. For isolated, one-of inspections on a single lot, which may be your use case in auditing, I think a standard sample size with confidence levels and margin of errors yielding a confidence interval on your estimate is more transparent. You can see this yourself by running some confidence level / margin of error pairs on the sampling sizes suggested by the standard. The errors get very large on every single inspection round, as you would expect when sampling say 5% from small populations. The idea of the standard is that by repeating the sampling on many subsequent lots the acceptance and rejection numbers become good estimators on whether the total supply of many lots is within its target AQL. This is basically expressed in Figure 1 with all the switching logic from inspection levels: the standard is designed for repeated (lot-by-lot) inspection, not for one-of exercises.