I have a report being generated from some transaction data (basically a list of items that should have been purchased) and I can also randomly sample the true transactions as they happen as an audit of the reporting process.

If I have a report with 1000 items included (today's reported sales) and during today I randomly sampled 50 real transactions and found that only 48 of them were in the report, is there a robust way to scale the error in this sampled audit to a full uncertainty on the accuracy of the 1000 item report?

Given that the report accuracy (or completeness) on the sample is 48/50 = 0.96, the naive approach would be to assume +/- 4% on the full 1000 item report, but of course the 48/50 came from a probabilistic process and given another random sample maybe the accuracy would have been 47/50 or 50/50.

Any advice would be much appreciated!

  • $\begingroup$ How do you select your random sample from the transactions? $\endgroup$ – RoryT Jun 22 '17 at 4:52
  • $\begingroup$ It is sometimes what I'd consider to be a proper random sample (random times) other times it is a concentrated time window - i.e. Perhaps 50 consecutive transactions in real time. Does that change the answer? $\endgroup$ – anthr Jun 22 '17 at 4:57
  • $\begingroup$ Yes, it could matter. For example if you sampled 50 transactions by picking a particular hour and auditing all transactions that occurred that hour, then that would be different to selecting a uniform random sample of 50 particular transactions. $\endgroup$ – RoryT Jun 22 '17 at 5:32
  • $\begingroup$ So the general way you select a transaction in your sample is you pick particular random time and then audit the next transaction after that time? $\endgroup$ – RoryT Jun 22 '17 at 5:35
  • $\begingroup$ Oh no sorry, the audit is completely separate from the reporting process, it is simply an attempt to validate the full report. The report is generated via code that is always running (but potentially not 100% accurate) - the sample is done in parallel and ideally all 50 sampled transactions would be present in the report. What I'm hoping is that we can use the sample to quantify the accuracy of the report. $\endgroup$ – anthr Jun 22 '17 at 5:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.