I want to evaluate my classifier model (Facebook FastText) against the ground truth. My dataset has two labels let's say A and B, so I have a binary classifier model. After the training (train and test split 80/20 or 70/30) I get my P,R i.e. the Precision, Recallvalues at k (P@k and R@k).

My Ground Truth (of a given size) has a lack of a label i.e. I have only the label A, but any samples of the label B i.e. in the ground truth I have correct labeling (by human expert)for A, and any labeling for B. Hence I can estimate against the ground truth the error rate for the label A, but not for the label B, because of missing samples.

Assumed that I have for the TestSet the correct evaluation of P and R, is there a way to estimate (worst/best case) the error rate for the B label as well?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.