1
$\begingroup$

Let me give an example. Assuming I have a classification technique for automatically measuring if a part in a car needs repair or not, it is my goal to check my classifier results against the ground truth of a test set. However, in my test set there are some items for which I cannot tell whether they need repair or not.

Obviously, I could exclude those items from my test set. However, I may introduce some bias by doing so. There are also some other unrelated constraints that prevent me from getting different test data. Is there a way to calculate a confidence interval or something similar? Ultimately, I would like to calculate accuracy, precision, and recall.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.