I work in a classification problem where collecting instances (labeled or unlabeled) is very expensive. In fact, labeling instances is cheap.

I want to explore an active learning strategy where I do not have access to unlabeled instances, but I can still ask for labeled instances by specifying:

  • Statistics on the attributes on the new instances (e.g. give me an instance with an age > 45)
  • The instance label (e.g. give me a new instance of label $l$, where $l \in L$ is one of the classification labels) or the label sampling distribution itself.

My questions are:

  • Does this problem have a name? It looks like a specific case of Active Learning, but I am not sure, since in Active Learning one starts with a set of unlabeled instances, which is not my case.

  • What types of approaches (from the most rudimentary to the more sophisticated) can I employ to identify the most informative sampling distribution from instance attributes or instance labels?

  • 1
    $\begingroup$ The first point can be partially addressed by membership query synthesis, since the strategy generates the instance from scratch. $\endgroup$
    – dawid
    Commented May 15, 2014 at 19:35

1 Answer 1


Your question is very closely related to this one: http://www.csresearchers.com/?qa=10/reverse-learning-labelling-expensive-instance-gathering

In your setting, unlabelled instances are costly to obtain, while it is near-free to get their labels. To address this, you can use active learning by formulating queries with synthesized data, this is called "membership queries synthesis". Basically, you can generate an instance (with no cost) in an uncertain region of the feature space and ask the labeller for its label, as described in the above link.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.