I want to find out the number of tests necessary to validate certain performance characteristics of an object detection system with a certain confidence. The system just detects if a relevant object is present or not (no classification etc.).
My system is supposed to have a true positive rate (TPR) of at least 90 % and a false positive rate (FPR) of maximum 5 %. Assume that the system can only detect one object under one set of environment conditions and is only supposed to be tested on this task. I want to test if the system fulfills those performance characteristics and give some metric of confidence about this test showing that it does fulfill them. How would I calculate the number of tests necessary as a function of the metric of confidence?
How could the same question be answered if I have m different objects to recognize under n different environment conditions?
