I want to evaluate my classifier model (Facebook
FastText) against the ground truth. My dataset has two labels let's say
B, so I have a binary classifier model. After the training (train and test split 80/20 or 70/30) I get my P,R i.e. the
k (P@k and R@k).
Ground Truth (of a given size) has a lack of a label i.e. I have only the label
A, but any samples of the label
B i.e. in the ground truth I have correct labeling (by human expert)for
A, and any labeling for
Hence I can estimate against the ground truth the error rate for the label
A, but not for the label
B, because of missing samples.
Assumed that I have for the
TestSet the correct evaluation of P and R, is there a way to estimate (worst/best case) the error rate for the
B label as well?