Say we have unlimited unlabeled data and we can ask an oracle for labels. We can use active learning to choose the most informative data samples for labeling, thus minimizing data labeling cost. If we choose a single sample at a time, we can choose the one with the highest class entropy.
How do we go about choosing a batch of
k samples to be labeled, e.g. for deep learning?
Some approaches I encountered are:
ksamples with the highest class entropy.
- As in 1., but ensure that there are the same amount of samples from every class in the batch.
Ksamples with the highest class entropy and randomly choose
ksamples out of them.
Are there any more sophisticated approaches that work in every scenario? Which of the above would be the best to use?