Given that I have an algorithm that classifies data points as 'true' or 'false'. and I want to estimate its FPR, FNR. It is not a supervised model where I start with a large training set of labeled data, so I thought of getting a sample data set and labeling it manually. Labeling is very laborious, so I want the sample set to be as small as possible, what is the formula to calculate the sample size (and obviously, what parameters will it require)?
In particular, I don't have a good estimate about the real ratio of 'true' vs. 'false' in the real world.
Are there better measures than FPR / FNR in such a case?