I have a bag of words binary text classification task. The SGD algorithm performed well for a certain target where number of labeled cases for training reached tens of thousands. For another target only about a thousand of labeled cases is available.

Colleagues can help me to manually label any additional case to the new target. Thus the "assisted" learning comes to mind. Number of unlabeled cases we have is virtually unlimited. The limitation is that colleagues can only label a few hundreds of cases at most so we have to choose them wisely.

How to chose cases to label with most impact to the precision of the classifier?

I recall reading a text online about choosing the cases with the highest uncertainty of classification or something but can't find it now.

Could you point me to any algorithm or a just some starting point to implement the above approach?

  • $\begingroup$ Please edit your question to describe the context of this problem. For example, what is the goal of this project, what sorts of text are you trying to classify, and what performance have you been able to achieve with the training data you already have? $\endgroup$ – Kodiologist Jun 12 '16 at 12:09

Here is an example implementation in sklearn. It's name "Label Propagation digits active learning" did not sound like what I am trying to do so I have missed it at first. Another name for my task found in the literature is "active adaptive learning".


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.