Questions tagged [active-learning]

Active learning is a setting where an automated learning system can request labels from an external source, perhaps a human user or a real-world experiment. It is used to try to learn good models while minimizing the number of interventions required.

Filter by
Sorted by
Tagged with
1
vote
1answer
17 views

Is active learning with decision trees possible?

Background to make sure I understand decision trees: In order to create a decision tree, we go through each dimension and we attempt create two partitions of the data by trying every possible "...
1
vote
0answers
68 views

Active learning with reinforcement learning: when to get labels in unlabeled streaming data

I have some realtime unlabeled sensor streaming and would like to decide which sample to label to improve the model performance. I would like to design the policy using reinforcement learning, instead ...
0
votes
1answer
20 views

How to get boundary points that are at interface of differnet classes in multilabel classification dataset

I am working on finding points which are at boundary of different classes. In other words finding points on which a classifier would be most confused or uncertain about. For a setting like multi label ...
0
votes
0answers
9 views

metric for unbalanced subsets in an active learning scenario

I am looking for a metric which measure "How unbalanced" a data subset is BUT since I will use it in an active learning scenario which means that my subset might not only unbalanced but also might not ...
0
votes
0answers
16 views

Specifiying threshold value for uncertainty sampling approach – active learning, machine learning

I need to implement a stream based uncertainty sampling mechanism which uses random forest as a classifier. Since I would like to improve my model (with the smallest possible amount of data) I would ...
1
vote
1answer
15 views

Active learning with a unlabelled pool - standard references & model-based labelling of the pool?

I'm looking into active learning for a multi-class classification problem, where there is a large pool of unlabelled data. I start out with a small set of labelled data and can labelled some more of ...
0
votes
1answer
49 views

Different terminologies for ground truth

It's general theory that a supervised learning approach can only get as good an accuracy as the quality of the ground truth. The ground truth is considered the best possible annotation. However, ...
1
vote
0answers
17 views

Active data acquisition with small datasets / Active Design of Experiment [closed]

Validated, I am facing a problem at work and I hope to find some light here, I have long worked with numerical methods, simulations and data analysis, but only recently I started studying and working ...
0
votes
1answer
19 views

Notation in definition of a quantity involving uncertainty and posterior probability

As a probs & stats noob, I'm still getting confused by notation. I would appreciate if someone could elaborate a bit on what's going on here, on p12 of Settles' Active Learning survey (2012). He ...
0
votes
1answer
52 views

Model selection and error estimation with active learning

I'm working on a classification problem where we have plenty of unlabelled data but it is costly and time-consuming to label them. So, I'm looking into active learning to economize on how much data we ...
1
vote
0answers
18 views

What is the convergence criteria of a semi-supervised learning algorithm

I would like to know about when to stop doing semi supervision? for example, I learn a classifier from a small dataset, and then I use it to label a pool of unlabelled dataset. How long should this ...
0
votes
0answers
24 views

Using active learning to polish the accuracy after learning from a dataset

I am trying to create a prediction model which would return a numerical score upon receiving a set of representative numbers. The purpose was to replicate human quality assessment procedure upon ...
1
vote
0answers
105 views

Active learning for object detection- Batch Selection

I have a small dataset of about 220 images for three classes. I am using YOLO (you only look once) network for an object detection. I am trying to use Active learning in order to reduce the number of ...
1
vote
1answer
496 views

Active Learning with Human-in-the-Loop

I did a lot of research and can't find a satisfactory answer. I have just a quick question about Active Learning and would be pleased if you could answer it. I'm still wondering if active learning ...
7
votes
3answers
163 views

Using ML to assist human labelling in dataset with highly unbalanced classes

Are there scientific issues with using ML to assist human annotation? I've got a 3 class unlabelled dataset where only 1 in 500 elements belong to the 2 classes of interest. The labels arn't ...
3
votes
3answers
1k views

Strategies for incorporating feedback for a ML algorithm

I am developing a text classification problem, in which at some time points, say at the end of each week, I receive a batch of feedback from users about correctly and wrongly classified inputs. I am ...
1
vote
2answers
41 views

Learning from Mistakes in a classification task

Let's say I have a text classifier that I have trained on some training dataset. Now when I run this trained classifier on some test dataset, I identify the cases where it went wrong (assuming I have ...
1
vote
1answer
108 views

Are there any tutorial for text categorization with Active Learning?

I am going to categorize texts into predefined topics. It looks like Active Learning approach is suits for me. Are there any good tutorials or advices?
0
votes
0answers
42 views

Are there clustering algorithms which require a human as an oracle?

Are there data clustering algorithms which by definition require human judgement as part of the algorithm? What I mean is that once in a while the algorithm presents the human operator (or any other ...
5
votes
1answer
112 views

Combine reinforces and unsupervised learning?

I have an existing set of data and plan to generate more data that follows the same pattern. To do this, I plan to use unsupervised learning. How can I provide feedback on the generated data and ...
1
vote
2answers
355 views

Active learning using logistic regression

I've an unlimited amount of unlabelled data and an oracle that is able to label portions of this data using only two labels "1" or "0". I want to ask a few number of questions to the oracle and after ...
2
votes
0answers
71 views

Active learning system confusion matrix

A colleague of mine has been developing a machine learning system with an active learning component. I was having trouble reproducing the metrics he's been reporting, until I found out that he's ...
1
vote
0answers
33 views

What description fits this this learning scenario, is this a form of active learning?

A system is involved in dialog with a human partner. The system has a model of a problem domain and knows mappings between words an concepts. The human has its own model of the problem domain and ...
3
votes
1answer
704 views

Batch Active Learning for classification?

Say we have unlimited unlabeled data and we can ask an oracle for labels. We can use active learning to choose the most informative data samples for labeling, thus minimizing data labeling cost. If we ...
1
vote
0answers
105 views

Active Learning Regression Problems [closed]

I Want to apply active learning in regression problems, but all the frameworks that I found are only for classification problems. Does someone know some framework for active learning in regression ...
1
vote
0answers
81 views

Is a test set sampled after training examples have been removed by an active learning algorithm valid?

In this case desire is to optionally label a final test set after training. The training is to be done using an active learning approach which can be biased towards different characteristics (...
3
votes
0answers
1k views

R package for active learning and sampling [closed]

I tried to find R package for active learning/sampling and stumbled over "activelearning" package. However, this package is not available on CRAN. Does someone know how good is this package? Are ...
5
votes
1answer
72 views

Labeling a pool of unlabelled samples iteratively

Problem setting I'm faced with a problem in which we have a large set of data points (100K), all of which are still unlabelled. These are to be used as input to a binary classifier at a later point ...
2
votes
1answer
393 views

Asymmetrical selective sampling for linear classification

I've got a online classification problem where I predict a class label {+1, -1} for an object and then show it to a user to get a real label. My task is to minimize ...