I have a collection of images with labels (such as "goat") that I'd like to train a classifier with, using a convolutional neural network or similar technique.
However, while the presence of a label implies presence of the feature, absence of a label does not imply absence of the feature. For example, an image labeled "goat" should have a goat somewhere in the image, but an image without the "goat" label might still have a goat. Perhaps it was a human-curated data set where the second person didn't think to label the goat.
This makes the error difficult to calculate: If my model outputs a probability that an image contains a goat, then I can calculate the error in the positive case (expected = 100%), but not in the negative case (expected = 0% would be incorrect).
This seems to be a ''Supervised Learning'' and ''Classification'' problem, but the techniques I've read about assume that absence of a label implies absence of the feature. Is there a name for this problem, or a reference for how to handle it?