Active learning with a unlabelled pool - standard references & model-based labelling of the pool?

Question

I'm looking into active learning for a multi-class classification problem, where there is a large pool of unlabelled data. I start out with a small set of labelled data and can labelled some more of the large pool of unlabelled data. It is also very cheap to run predictions on the unlabelled data. I've seen suggestions that active learning can select unrepresentative samples (usually outliers that the model is very unsure about in some sense), which may be a concern.

Is there some standard literature what one should do about this? In particular, I've seen some papers that seem to (temporarily - i.e. may get "human-labelled" later, but for the next iteration of the model we may treat them as labeled, even if it's only on the basis of a model prediction) assign labels to some of the unlabelled based on the model predictions (e.g. if the probabilitiy of a class is >0.9). Or, perhaps you assign soft-labels, i.e. instead of 0 or 1, for a class you assign the model predictied probabilities and train against those. This is sort of a semi-supervised approach.

What I've failed to find is some general theory or higher level discussion of what is generally likely to work. What are the obvious references/papers/books that discuss these topics? I'm aware of a bunch of papers like the classic Active Learning with Statistical Models and was looking at e.g. Cost-effective active learning for deep image classification. I realize there's a ton of literature on this type of stuff out there, but what are the key things I should look at (even better if you are willing to briefly summarize them)?

Answer 1

There has been some critique to standard pool-based query algorithms (like QBC and Uncertainty sampling) in this regard. These standard methods don't take the density of the input distribution into account while querying data. Hence, they might end up querying outliers.

On the other hand, pure density-based algorithms (like e.g. random sampling) end up querying redundant data. The necessity of a tradeoff between density (or diversity) of data to query and uncertainty about them has been noted in many papers, see e.g. this paper. A nice algorithm that tries to combine the advantages of both is Stochastic Query by Forest.

However, this problem has been addressed only intuitively/empirically. I think that a theoretical framework to quantify this tradeoff is still missing. Maybe, the closest result to such a theoretical understanding is in this preprint. Here, the authors suggest to quantify the diversity/density by employing the Wasserstein distance. I think that equation (2) in this paper is the best existing formalisation of this intuition.