Active Learning with Human-in-the-Loop

Question

I did a lot of research and can't find a satisfactory answer. I have just a quick question about Active Learning and would be pleased if you could answer it.

I'm still wondering if active learning only fit for the training of a classifier? I know it can help to reduce the size of the training data while iteratively learning from an unlabeled data pool using human annotation. But all papers and literature I could found refer only to the training phase of classifiers.

Is it also possible if the classifier is "live" and make some predictions to use the new observations to actively learn from them? Like continuously active learning? For example, if the classifier has a low confidence about a prediction (wrong prediction), can this new information be used to improve the classifier?

In this context, I've read a lot about Human-in-the-Loop with active learning. https://www.slideshare.net/BillLiu31/natural-intelligence-the-human-factor-in-ai If I look at this presentation, then I would said yes, it can continuously learning actively. Facebook's DeepFace is described, where the algorithm ask a human to help with labeling if it is uncertain about the face (page 63). Or by adding a human feedback loop on page 82.

I also found another real-world example where Coca Cola claims to use active learning. A user can correct invalid predictions and the algorithm could be improved over time via active learning (sounds similar to DeepFace): https://developers.googleblog.com/2017/09/how-machine-learning-with-tensorflow.html

In fact, I have found some more real-world example where I believe or the companies say that they use active learning. For example, at figure-eight. But nobody says how they implement it. Its very difficult to find good literature or papers about this intention.

Answer 1

The answer is yes. Empirically, if a classifier predicts on a new sample, is determined to be wrong by humans, that data point can be used to fine-tune the classifier. It effectively becomes additional training data. You have to be reasonable though in your expectations. The training algorithm is no different than before, say, some kind of batch gradient descent, so you still need a significant number of samples to make a meaningful impact on the classifier's accuracy on future samples. For example, you can wait for 100 wrong predictions, and then feed those into your classifier via fine-tuning at some small learning rate. This will marginally improve it's accuracy on that category in the future. You could in theory just update it after every single prediction, but this could conceivably result in poor performance due to small batch size.

Also, having "low confidence" in a prediction is a bit of a misnomer. The classifier doesn't know that it's wrong but objectively, it thinks that whatever category you're interested in is not present in the sample presented. It's not like a human being who would be like "oh you're right, there really is a raccoon in this picture, right there" after having the raccoon pointed out.The issue with this logic is that it's not enough to suddenly change your mind on the given sample, the question is, whether you'll be able to spot similar samples in the future. If the raccoon was really difficult to spot in the first place, the marginal gain of having a single example pointed out to you might be quite small.