I did a lot of research and can't find a satisfactory answer. I have just a quick question about Active Learning and would be pleased if you could answer it.
I'm still wondering if active learning only fit for the training of a classifier? I know it can help to reduce the size of the training data while iteratively learning from an unlabeled data pool using human annotation. But all papers and literature I could found refer only to the training phase of classifiers.
Is it also possible if the classifier is "live" and make some predictions to use the new observations to actively learn from them? Like continuously active learning? For example, if the classifier has a low confidence about a prediction (wrong prediction), can this new information be used to improve the classifier?
In this context, I've read a lot about Human-in-the-Loop with active learning. https://www.slideshare.net/BillLiu31/natural-intelligence-the-human-factor-in-ai If I look at this presentation, then I would said yes, it can continuously learning actively. Facebook's DeepFace is described, where the algorithm ask a human to help with labeling if it is uncertain about the face (page 63). Or by adding a human feedback loop on page 82.
I also found another real-world example where Coca Cola claims to use active learning. A user can correct invalid predictions and the algorithm could be improved over time via active learning (sounds similar to DeepFace): https://developers.googleblog.com/2017/09/how-machine-learning-with-tensorflow.html
In fact, I have found some more real-world example where I believe or the companies say that they use active learning. For example, at figure-eight. But nobody says how they implement it. Its very difficult to find good literature or papers about this intention.