Method to classify and recode high and low probability regions?

Question

Most classification methods typically train and predict over all instances, possibly giving a less efficient answer. To give an example, imagine two randomly populated regions on a 2D grid (both populated with the same two binary classes). One well defined space (let's say top half) has 80% of positive classes, and the other only 50%. Further, assume these probabilities are stable.

I envision some type of learner like a classification tree could identify the regions using entropy based splitting. But rather than merging more noisy sets, I would want to literally prune or recode the more noisy regions to a 3rd class as a method to pre-process for the prediction step. So that rather than getting a low average prediction across all of the exemplars, my learner could focus more on high regions of probability. Does anyone know of such an existing method?

Answer 1

This is exactly the focus of active learning. Do watch this video by Sanjoy Dasgupta, John Langford: http://videolectures.net/icml09_dasgupta_langford_actl/ to understand the framework. In a way, it combines unsupervised learning(understanding the locality structure) with semi-supervised learning and label propagation.