I want to find out the number of tests necessary to validate certain performance characteristics of an object detection system with a certain confidence. The system just detects if a relevant object is present or not (no classification etc.).

My system is supposed to have a true positive rate (TPR) of at least 90 % and a false positive rate (FPR) of maximum 5 %. Assume that the system can only detect one object under one set of environment conditions and is only supposed to be tested on this task. I want to test if the system fulfills those performance characteristics and give some metric of confidence about this test showing that it does fulfill them. How would I calculate the number of tests necessary as a function of the metric of confidence?

How could the same question be answered if I have m different objects to recognize under n different environment conditions?

  • $\begingroup$ As it as easy as testing 1000 times for a precision of +-0,1 %? $\endgroup$ – hcl734 Nov 26 '20 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.