Let me give an example. Assuming I have a classification technique for automatically measuring if a part in a car needs repair or not, it is my goal to check my classifier results against the ground truth of a test set. However, in my test set there are some items for which I cannot tell whether they need repair or not.

Obviously, I could exclude those items from my test set. However, I may introduce some bias by doing so. There are also some other unrelated constraints that prevent me from getting different test data. Is there a way to calculate a confidence interval or something similar? Ultimately, I would like to calculate accuracy, precision, and recall.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.