1
$\begingroup$

I want to evaluate my classifier model (Facebook FastText) against the ground truth. My dataset has two labels let's say A and B, so I have a binary classifier model. After the training (train and test split 80/20 or 70/30) I get my P,R i.e. the Precision, Recallvalues at k (P@k and R@k).

My Ground Truth (of a given size) has a lack of a label i.e. I have only the label A, but any samples of the label B i.e. in the ground truth I have correct labeling (by human expert)for A, and any labeling for B. Hence I can estimate against the ground truth the error rate for the label A, but not for the label B, because of missing samples.

Assumed that I have for the TestSet the correct evaluation of P and R, is there a way to estimate (worst/best case) the error rate for the B label as well?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.