In this case desire is to optionally label a final test set after training. The training is to be done using an active learning approach which can be biased towards different characteristics (indecision, confidence, etc).
Would the test set in this scenario be useful at all? For example, if the active learning algorithm was biased to select high confidence samples the remaining data would have a different mix of data than what was started with (less positives and less high confidence data). Would totally invalidate the test set or can you still draw valid conclusions from it?