I want to compare the success rate of two configurations of a program. A program run two times on the same set of photographs and returns each time a list of face matches.
The success rate is computed for the two run with
e.g. Configuration A : 4500 success over 5000 matches Configuration B : 4400 success over 4900 matches
The two-proportion z-test would have been a good candidate, but it make the assumption that "The sampling method for each population is simple random sampling." which does not look apply to my data.
- What would be a good way to test that a given configuration success rate is significantly higher than another one ?
- Should I use a one-tailed test in this situation, or would a two tailed test be suitable ?