Biased Data in Machine Learning

Question

I am working on a Machine Learning project with data that is already (heavily) biased by data selection.

Let's assume you have a set of hard coded rules. How do you build a machine learning model to replace it, when all the data it can use is data that was already filtered by those rules?

To make things clear, I guess the best example would be Credit Risk Assessment: The task is to filter all clients that are likely to fail to make a payment.

• Now, the only (labeled) data you have are from clients that have been accepted by the set of rules, because only after accepting you will see if someone pays or not (obviously). You don't know how good the set of rules is and how much they will affect the payed- to not-payed distribution. Additionally, you have unlabeled data from the clients that have been declined, again because of the set of rules. So you don't know what would have happened with those clients if they had been accepted.

E.g one of the rules could be: "If age of client < 18 years, then do not accept"

The classifier has no way to learn how to handle clients that have been filtered by these rules. How is the classifier supposed to learn pattern here?

Ignoring this problem, would lead to the model being exposed to data it has never encountered before. Basically, I want to estimate the value of f(x) when x is outside [a, b] here.

Answer 1

You are right to be concerned - even the best models can fail spectacularly if the distribution of out-of-sample data differs significantly from the distribution of the data that the model was trained/tested on.

I think the best you can do is train a model on the labelled data that you have, but try to keep the model interpretable. That probably means only being limited to simple models. Then, you could attempt to reason how the rules learnt by your model might interact with the prior rules you had, in an attempt to estimate how well your model might work on the unfiltered population.

For example - suppose, your model finds that in your labelled dataset, the younger the client is, the more likely they were to default. Then it may be reasonable to assume that your model will work well if you removed the prior filter of "If age of client < 18 years, then do not accept".

Answer 2

I'm not sure I entirely understand that question, but so far as I understand it you're asking how to train a classifier to predict on samples lying outside the domain of the samples it has already seen. This is, generally speaking and so far as I know, not possible. Machine learning theory is based on the idea of "empirical risk minimization," which boils down to assuming that your training set is a good approximation of your true distribution over samples and labels. If that assumption is violated, there aren't really any guarantees.

You mention unlabeled data -- I don't know if this would solve your problem, but semi-supervised learning has many methods for trying to learn classifiers given both labeled and unlabeled data, and you may want to consider looking into those (for example, transductive SVMs).

Answer 3

Your rules may give you a way to perform data augmentation. Copy a positive sample, change the age to 17, and then mark it as a negative sample.

This procedure won't necessarily be trivial or useful for all datasets. I work with NLP data and it's tricky to do well in that domain. For example, if you have other features correlated with age, you may end up with unrealistic samples. However, it provides an avenue to expose the system to something like the samples that didn't make it into the dataset.

Answer 4

One thing that has worked for us in a similar situation is doing a bit of reinforcement learning (explore and exploit). On top of the rule based model, we ran a explorer which would with a small likelihood change the response of the model, so in occasional cases where the model would not recommend a card to a 17-year old, the explorer would overturn the model's decision and issue a card. From these occasional cases you would generate learning data for a future learning model where it can be used to decide to recommend cards for 17 year olds based on if the ones that were issued to 17 year olds by the explorer did not default and so you can build systems that can work outside the biases of your existing model.

Answer 5

From a practical standpoint it is difficult/unreasonable to ask a model to predict something on cases that are not possible in the current system (no free lunch).

One way to circumvent that problem is to add randomization to the current (deployed) system, e.g. to add the possibility to bypass (some of) the rules with a small, controlled probability (and hence a predictable cost).

Once you managed to convince the people responsible for the system to do that then you can use off-policy evaluation methods like importance sampling to ask "what-if" questions. E.g. what would be the expected credit risk if we would allow people that are currently dropped by the rules to take a credit. One can even simulate the effect of your (biased) prediction model on that population. A good reference for that kind of methods is Bottou's paper on counterfactual learning and reasoning.

Answer 6

The classical statistical answer is that if the selection process is in the data and described by the model or selection is at random then the parametrical model contemplates it correctly. See Donald Rubin paper Inference and Missing data (1976). You do need to include the mechanism of data selection in your model. This is a field where parametric inference should do better than pure machine learning.

Answer 7

This is akin to the after-life dilemma: what ratio of good and bad deeds (data) is sufficient to get to heaven instead of hell (class), after one dies (filter!). Herein, death serves as the filter, leading to missing values towards a supervised learning scheme.

I want to disambiguate between missing-value problem and 'biased data' problem. There is no such thing as biased data, there is such a thing as 'biased model' explaining said data, but the data itself isn't biased, it is merely missing. If the missing data is meaningfully correlated to observable data, then it is entirely possible to train an unbiased model and achieve good predictive results.

If the missing data is completely uncorrelated with observable data, then its a case of 'you don't know what you don't know'. You can use neither supervised, nor unsupervised learning methods. The problem is outsides the realms of data science.

Therefore, for the sake of meaningful solution, lets assume that missing data is correlated with observable data. We'll exploit said correlation.

There are several data mining algorithms that attempt to solve such a problem. You can try 'Ensemble methods' like Bagging-n-Boosting or 'frequent pattern mining' algorithms like Apriori and FP-growth. You can also explore methods in Robust Statistics.