Effects of selection bias in training data introduced by previous model outputs

Question

I am developing a random forest for a binary classification problem where the trained data is heavily skewed towards one class (90% is class A and 10% is class B). The model scores data points based on their likelihood to be of class A, i.e., the number of trees in the forest that classify the data point as class A. This scoring effectively ranks data from the point that is most likely to be class A (top of the list), to the least likely (bottom of the list).

The training data is generated by another system on a monthly basis, and human operators review the data labeling each point as A or B. Suppose this model is used in a business process in such a way that the top 20% of the scored list (most likely to be A) is automatically labeled as A, and the remaining 80% is manually labeled as usual. This is going to introduce some errors in the automatically generated labels, and possibly more and more if we consider for automatic labeling increasing portions of the list (30% and above). However, if the model is good enough, we can live with such errors that define acceptable thresholds of precision, recall, accuracy, or whatever performance metric.

Since the manual labeling process typically changes over time, the model needs to be retrained every now and then. However, the training data will progressively contain less and less "obvious" examples of class A, because they will be excluded from the manual labeling process – effectively introducing a selection bias in the training data. This is of course unless we want to also learn from the previous model outputs (the 20% of data points labeled automatically), which intuitively doesn't sound very smart.

How do you expect the model to perform with such a selection bias in training data? What could happen if, rather than excluding from manual labeling 20% of data, we exclude 70% or 80%? The question might sound a little too generic, but this seems to me an interesting theoretical problem closely related to the application of Machine Learning approaches to real-world problems. I have never stumbled upon any paper about this. Maybe it also probably has a specific name, better than the generic "selection bias in training data", but I don't recall seeing this in the past.

Answer 1

This approach could in principle lead to problems, but those might be mitigated somewhat by use of random forest. In any event, it sounds like you might be able to get the data you need to evaluate the tradeoffs for your particular application.

Note that some recommend under-sampling of the majority class to minimize classification bias toward the majority class; see this answer for some discussion. So under-sampling the A class is not a problem in itself. The approach you describe does have 2 potential problems: throwing out true B-class members that happen to score in the top 20% (or top 30% or ...) along with the correctly classified A-class members, and a bias against certain predictor variable values or combinations that characterize those cases with the highest A-class probability.

Whether B-class members are being lost in that top 20% depends on the nature of the Receiver Operating Characteristic curve. As you have a rank-listing of probabilities along with true (manual) classifications, you can construct that curve directly yourself and evaluate its implications for losing B-class members in that top 20%. You don't discuss the relative costs of different misclassifications, but if you mostly care about identifying most B-class members (as I infer) and any substantial fraction of B-class members are included in that top 20% then I would be worried.

If the top 20% almost never include B-class members, then the issue is more subtle: whether you are selecting against particular characteristics of A-class members in a way that hurts the overall delineation of A from B. In some types of modeling approaches that might be a big problem, but the random choice of a small number of predictors at each node in constructing a random forest might make that less of an issue in your application. Again, you may have access to data needed to evaluate this directly. For example: start with a completely annotated data set, make a bootstrapped collection of samples from those data, develop models from each of those bootstrap samples with different choices for the highest (A-class) score cutoff for inclusion, and see how well they perform (with a proper scoring rule) in aggregate at different cutoffs in modeling the original data set.

Informally, the approach you describe might be thought of as applying a type of boosting to random forests.* That is, the boosting process can be thought of as sequentially fitting to the residuals from the earlier versions of a model, so if you think about the bottom 80% as being "residuals" then that is sort of what this approach is doing. In this analogy to boosted trees, however, note that you don't just throw away the earlier versions in true boosting; rather, you gradually add in the additional trees developed from fitting the residuals. So in your case you might want to consider gradually adapting your prior model rather than completely replacing it when new manually annotated data are available. Again, you probably have the data you need to see how well that would work.

I am, however, troubled by your statement that "the manual labeling process typically changes over time." I'm not quite sure what that means for you in practice, but if the ground truth of the labeling is changing it's not clear exactly what you are classifying. That could be the biggest problem of all.

---

*Although random forests and boosted trees are different, a quick web search did find a proposal to boost random forests. This isn't my specialty; there may be other similar proposals.